Bug 715512

Summary: RepoError when sync was "in-progress" for two repos.
Product: Red Hat Update Infrastructure for Cloud Providers Reporter: Sachin Ghai <sghai>
Component: RHUAAssignee: Pradeep Kilambi <pkilambi>
Status: CLOSED CURRENTRELEASE QA Contact: wes hayutin <whayutin>
Severity: high Docs Contact:
Priority: unspecified    
Version: 2.0CC: jason.dobies, kbidarka, sghai, tsanders
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-05-31 12:54:10 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 711223    

Description Sachin Ghai 2011-06-23 07:07:00 UTC 
Description of problem:

I started the sync for two repos and got following error in pulp.log:


2011-06-23 11:44:52,660 2890:140150922262288: pulp.server.api.repo_sync:INFO: repo_sync:211 Sync invoked for repo <rhel-server-6-optional-releases-6Server-x86_64>
2011-06-23 11:44:52,663 2890:140150922262288: pulp.server.api.repo_sync:INFO: repo_sync:221 Sync of rhel-server-6-optional-releases-6Server-x86_64 starting, skip_dict = {}
2011-06-23 11:44:52,664 2890:140150922262288: pulp.server.api.synchronizers:INFO: synchronizers:390 cacert = </etc/pki/content/rhel-server-6-optional-releases-6Server-x86_64/feed-rhel-server-6-optional-releases-6Server-x86_64.ca>, cert = </etc/pki/content/rhel-server-6-optional-releases-6Server-x86_64/feed-rhel-server-6-optional-releases-6Server-x86_64.cert>

2011-06-23 11:44:52,669 2890:140150911772432: pulp.server.api.repo_sync:INFO: repo_sync:211 Sync invoked for repo <rhel-server-6-updates-6Server-x86_64>
2011-06-23 11:44:52,673 2890:140150911772432: pulp.server.api.repo_sync:INFO: repo_sync:221 Sync of rhel-server-6-updates-6Server-x86_64 starting, skip_dict = {}
2011-06-23 11:44:52,675 2890:140150911772432: pulp.server.api.synchronizers:INFO: synchronizers:390 cacert = </etc/pki/content/rhel-server-6-updates-6Server-x86_64/feed-rhel-server-6-updates-6Server-x86_64.ca>, cert = </etc/pki/content/rhel-server-6-updates-6Server-x86_64/feed-rhel-server-6-updates-6Server-x86_64.cert>
2011-06-23 11:46:25,938 2890:140150922262288: pulp.server.tasking.task:ERROR: task:380 Task failed: Task b53a5b78-9cef-11e0-a639-525400734a2e: _sync(rhel-server-6-optional-releases-6Server-x86_64, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7f77605c83d0>, skip_dict={}, max_speed=None, threads=None)
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/tasking/task.py", line 329, in run
    result = self.callable(*self.args, **self.kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 224, in _sync
    progress_callback, synchronizer, max_speed, threads)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 302, in fetch_content
    progress_callback, max_speed, threads)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 429, in sync
    report = self.yum_repo_grinder.fetchYumRepo(store_path, callback=progress_callback)
  File "/usr/lib/python2.6/site-packages/grinder/RepoFetch.py", line 393, in fetchYumRepo
    self.yumFetch.getRepoData()
  File "/usr/lib/python2.6/site-packages/grinder/RepoFetch.py", line 138, in getRepoData
    for ftype in self.getRepoXmlFileTypes():
  File "/usr/lib/python2.6/site-packages/grinder/RepoFetch.py", line 123, in getRepoXmlFileTypes
    return self.repo.repoXML.fileTypes()
  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1417, in <lambda>
    repoXML = property(fget=lambda self: self._getRepoXML(),
  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1413, in _getRepoXML
    raise Errors.RepoError, msg
RepoError: Cannot retrieve repository metadata (repomd.xml) for repository: . Please verify its path and try again

[root@dhcp201-111 ~]# 


Version-Release number of selected component (if applicable):
pulp 0.194
rhui-tools 2.0.32

How reproducible:
Its a intermittent issue.

Steps to Reproduce:
1. Add two repos and trigger the individual sync
2. keep checking the pulp logs

  
Actual results:
Got RepoError: Cannot retrieve repository metadata (repomd.xml) for repository: . Please verify its path and try again

Expected results:
Reos sync should run without any error.

Additional info:
-----------------------------------------------------------------------------
rhui (sync) => vr

Select a repository: 
  1  - Red Hat Enterprise Linux Server 6 Optional Releases (RPMs) (6Server-x86_64)
  2  - Red Hat Enterprise Linux Server 6 Updates (RPMs) (6Server-x86_64)
Enter value (1-2) or 'b' to abort: 1

  Start Time:     06-23-2011 11:44
  Finish Time:    06-23-2011 11:46
  Elapsed Time:   0:01:33
  Result:         Error
  Exception:      RepoError()
  Traceback:
Traceback (most recent call last):

  File "/usr/lib/python2.6/site-packages/pulp/server/tasking/task.py", line 329, in run
    result = self.callable(*self.args, **self.kwargs)

  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 224, in _sync
    progress_callback, synchronizer, max_speed, threads)

  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 302, in fetch_content
    progress_callback, max_speed, threads)

  File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 429, in sync
    report = self.yum_repo_grinder.fetchYumRepo(store_path, callback=progress_callback)

  File "/usr/lib/python2.6/site-packages/grinder/RepoFetch.py", line 393, in fetchYumRepo
    self.yumFetch.getRepoData()

  File "/usr/lib/python2.6/site-packages/grinder/RepoFetch.py", line 138, in getRepoData
    for ftype in self.getRepoXmlFileTypes():

  File "/usr/lib/python2.6/site-packages/grinder/RepoFetch.py", line 123, in getRepoXmlFileTypes
    return self.repo.repoXML.fileTypes()

  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1417, in <lambda>
    repoXML = property(fget=lambda self: self._getRepoXML(),

  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1413, in _getRepoXML
    raise Errors.RepoError, msg

RepoError: Cannot retrieve repository metadata (repomd.xml) for repository: . Please verify its path and try again
Comment 1 Sachin Ghai 2011-06-23 07:15:02 UTC 
Sync status is "Unknown" in rhui-manager.


------------------------------------------------------------------------------
             -= Red Hat Update Infrastructure Management Tool =-


-= Repository Synchronization Status =-

Last Refreshed: 12:31:13
(updated every 5 seconds, ctrl+c to exit)

Next Sync                    Last Sync                    Last Result         
------------------------------------------------------------------------------
Red Hat Enterprise Linux Server 6 Optional Releases (RPMs) (6Server-x86_64)
Unknown                      Never                        Error      

Red Hat Enterprise Linux Server 6 Updates (RPMs) (6Server-x86_64)
In Progress                  Never                        Never       


                                  Connected: dhcp201-111.englab.pnq.redhat.com
------------------------------------------------------------------------------




------------------------------------------------------------------------------
rhui (sync) => sr

Select one or more repositories to schedule to be synchronized before its scheduled time.
The sync will happen as soon as possible depending on other tasks that may be executing
in the RHUI. Only repositories that are not currently synchronizing are displayed.

         Last Result         Next Sync         Repository
         ------------------------------------------
Enter value (1-0) to toggle selection, 'c' to confirm selections, or '?' for more commands:
Comment 2 Sachin Ghai 2011-06-23 07:35:43 UTC 
It is not a intermittent issue. I got the same issue again.

This time I added small rhui1.2 repos and triggered the individual sync for all of them at once.


This is very initial screenshot. 


--------------------------------------------------------------------
Red Hat Enterprise Linux Server 6 Updates (RPMs) (6Server-x86_64)
In Progress                  Never                        Never       

Red Hat Update Infrastructure 1.1 (RPMs) (5Server-i386)
In Progress                  Never                        Never       

Red Hat Update Infrastructure 1.1 (RPMs) (5Server-x86_64)
In Progress                  Never                        Never       

Red Hat Update Infrastructure 1.2 (RPMs) (5Server-i386)
In Progress                  Never                        Never       

Red Hat Update Infrastructure 1.2 (RPMs) (5Server-x86_64)
Awaiting Execution           Never                        Never       

Red Hat Update Infrastructure 1.2 (SRPMS) (5Server-i386)
Awaiting Execution           Never                        Never       

Red Hat Update Infrastructure 1.2 (SRPMS) (5Server-x86_64)
06-23-2011 18:53             Never                        Never       
--------------------------------------------------------------------



     
Here you can see the error in "Last Result" for RHUI1.2 (SRPMS)



Next Sync                    Last Sync                    Last Result         
------------------------------------------------------------------------------
Red Hat Enterprise Linux Server 6 Updates (RPMs) (6Server-x86_64)
In Progress                  Never                        Never       

Red Hat Update Infrastructure 1.1 (RPMs) (5Server-i386)
06-23-2011 18:53             06-23-2011 12:55             Success    

Red Hat Update Infrastructure 1.1 (RPMs) (5Server-x86_64)
06-23-2011 18:53             06-23-2011 12:55             Success    

Red Hat Update Infrastructure 1.2 (RPMs) (5Server-i386)
06-23-2011 18:53             06-23-2011 12:55             Success    

Red Hat Update Infrastructure 1.2 (RPMs) (5Server-x86_64)
06-23-2011 18:53             06-23-2011 12:56             Success    

Red Hat Update Infrastructure 1.2 (SRPMS) (5Server-i386)
06-23-2011 18:53             Never                        Error      

Red Hat Update Infrastructure 1.2 (SRPMS) (5Server-x86_64)
06-23-2011 18:53             Never                        Never       


                                  Connected: dhcp201-111.englab.pnq.redhat.com
------------------------------------------------------------------------------
^Crhui (sync) => 




------------------------------------------------------------------------------
^Crhui (sync) => vr

Select a repository: 
  1  - Red Hat Enterprise Linux Server 6 Updates (RPMs) (6Server-x86_64)
  2  - Red Hat Update Infrastructure 1.1 (RPMs) (5Server-i386)
  3  - Red Hat Update Infrastructure 1.1 (RPMs) (5Server-x86_64)
  4  - Red Hat Update Infrastructure 1.2 (RPMs) (5Server-i386)
  5  - Red Hat Update Infrastructure 1.2 (RPMs) (5Server-x86_64)
  6  - Red Hat Update Infrastructure 1.2 (SRPMS) (5Server-i386)
  7  - Red Hat Update Infrastructure 1.2 (SRPMS) (5Server-x86_64)
Enter value (1-7) or 'b' to abort: 6

  Start Time:     06-23-2011 12:55
  Finish Time:    06-23-2011 12:55
  Elapsed Time:   0:00:08
  Result:         Error
  Exception:      RepoError()
  Traceback:
Traceback (most recent call last):

  File "/usr/lib/python2.6/site-packages/pulp/server/tasking/task.py", line 329, in run
    result = self.callable(*self.args, **self.kwargs)

  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 224, in _sync
    progress_callback, synchronizer, max_speed, threads)

  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 302, in fetch_content
    progress_callback, max_speed, threads)

  File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 429, in sync
    report = self.yum_repo_grinder.fetchYumRepo(store_path, callback=progress_callback)

  File "/usr/lib/python2.6/site-packages/grinder/RepoFetch.py", line 393, in fetchYumRepo
    self.yumFetch.getRepoData()

  File "/usr/lib/python2.6/site-packages/grinder/RepoFetch.py", line 138, in getRepoData
    for ftype in self.getRepoXmlFileTypes():

  File "/usr/lib/python2.6/site-packages/grinder/RepoFetch.py", line 123, in getRepoXmlFileTypes
    return self.repo.repoXML.fileTypes()

  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1417, in <lambda>
    repoXML = property(fget=lambda self: self._getRepoXML(),

  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1413, in _getRepoXML
    raise Errors.RepoError, msg

RepoError: Cannot retrieve repository metadata (repomd.xml) for repository: . Please verify its path and try again


------------------------------------------------------------------------------
rhui (sync) =>
Comment 6 Jay Dobies 2011-07-12 17:55:39 UTC 
Please let me know if this is still an issue with Pulp .204. This was likely fixed with the grinder changes to threading v. processes.
Comment 7 Sachin Ghai 2011-07-13 11:04:46 UTC 
I verified this with new builds ( pulp 0.204 and rhui-tools 2.0.37). This issue is not reproducible.

I started the sync for multiple repos at once. 

-= Repository Synchronization Status =-

Last Refreshed: 14:36:24
(updated every 5 seconds, ctrl+c to exit)

Next Sync                    Last Sync                    Last Result         
------------------------------------------------------------------------------
Red Hat Enterprise Linux Server 6 Optional Updates (RPMs) (6Server-x86_64)
In Progress                  07-12-2011 14:28             Success    

Red Hat Enterprise Linux Server 6 Releases (RPMs) (6Server-x86_64)
In Progress                  07-12-2011 01:02             Success     

Red Hat Enterprise Linux Server 6 Updates (RPMs) (6Server-x86_64)
In Progress                  07-13-2011 12:09             Success    

Red Hat Update Infrastructure 1.1 (RPMs) (5Server-i386)
07-13-2011 20:06             07-13-2011 14:34             Success    

Red Hat Update Infrastructure 1.1 (RPMs) (5Server-x86_64)
07-13-2011 20:06             07-13-2011 14:36             Success    

Red Hat Update Infrastructure 1.2 (RPMs) (5Server-i386)
Awaiting Execution           07-12-2011 14:30             Success    

Red Hat Update Infrastructure 1.2 (RPMs) (5Server-x86_64)
In Progress                  07-12-2011 14:28             Success    


                                  Connected: dhcp201-141.englab.pnq.redhat.com
--------------------------------------------------------------------------




Sync completed successfully for all repos, without this mentioned RepoError.


------------------------------------------------------------------------------
             -= Red Hat Update Infrastructure Management Tool =-


-= Repository Synchronization Status =-

Last Refreshed: 16:25:01
(updated every 5 seconds, ctrl+c to exit)

Next Sync                    Last Sync                    Last Result         
------------------------------------------------------------------------------
Red Hat Enterprise Linux Server 6 Optional Updates (RPMs) (6Server-x86_64)
07-13-2011 20:02             07-13-2011 14:37             Success    

Red Hat Enterprise Linux Server 6 Releases (RPMs) (6Server-x86_64)
07-13-2011 20:02             07-13-2011 16:22             Success    

Red Hat Enterprise Linux Server 6 Updates (RPMs) (6Server-x86_64)
07-13-2011 20:02             07-13-2011 14:40             Success    

Red Hat Update Infrastructure 1.1 (RPMs) (5Server-i386)
07-13-2011 20:06             07-13-2011 14:34             Success    

Red Hat Update Infrastructure 1.1 (RPMs) (5Server-x86_64)
07-13-2011 20:06             07-13-2011 14:36             Success    

Red Hat Update Infrastructure 1.2 (RPMs) (5Server-i386)
07-13-2011 20:06             07-13-2011 14:37             Success    

Red Hat Update Infrastructure 1.2 (RPMs) (5Server-x86_64)
07-13-2011 20:06             07-13-2011 14:36             Success    


                                  Connected: dhcp201-141.englab.pnq.redhat.com
------------------------------------------------------------------------------
^Crhui (sync) =>
Comment 8 wes hayutin 2011-08-01 21:39:32 UTC 
moving to release pending
Comment 9 wes hayutin 2012-05-31 12:54:10 UTC 
closing out, product released