Bug 1179463

Summary: Pulp work stays stuck at the waiting state
Product: [Retired] Pulp Reporter: Brian Bouterse <bmbouter>
Component: API/integrationAssignee: Brian Bouterse <bmbouter>
Status: CLOSED CURRENTRELEASE QA Contact: Preethi Thomas <pthomas>
Severity: high Docs Contact:
Priority: urgent    
Version: 2.5CC: dkliban, mhrivnak, pthomas
Target Milestone: ---Keywords: Triaged
Target Release: 2.5.2   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-02-05 21:47:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Brian Bouterse 2015-01-06 20:02:05 UTC
While testing the 2.5.2 RC on a system, bcourt noticed that tasks would stay at the waiting state and never move forward. Some pre-investigation was done and the symptom can be described like this:

A machine running pulp_workers that has a hostname which contains a 'd' 'q' or '.' specifically as the last character of the hostname will not have any tasks move beyond the waiting state. It results in pulp-admin giving the following output:

[pulp@bcourt-rhel6-prod ~]$ pulp-admin rpm repo sync run --repo-id rhel5
+----------------------------------------------------------------------+
                    Synchronizing Repository [rhel5]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.


[-]
Waiting to begin...

After some investigation it was determined this affects all 2.5.X releases but it does not affect 2.4.X releases or earlier. It also does not affect 2.6.X releases because the task dispatching system was overhauled for the 2.6.X release and that is free of this defect.

Comment 1 Brian Bouterse 2015-01-06 20:26:02 UTC
Assuming this lands in 2.5.2, I made the PR against 2.5-testing:

https://github.com/pulp/pulp/pull/1488

Comment 2 Brian Bouterse 2015-01-07 14:53:28 UTC
Merged to 2.5-testing -> 2.5-dev -> 2.6-dev -> master

Comment 3 Brian Bouterse 2015-01-07 19:31:39 UTC
QA: to verify this BZ you will need to do so on a machine whose hostname ends in 'd' or 'q'. For example 'pulp-prod' would work.

Comment 4 Preethi Thomas 2015-01-09 19:47:45 UTC
verified
[root@pulp-prod ~]# rpm -qa pulp-server
pulp-server-2.5.2-0.2.beta.el6.noarch
[root@pulp-prod ~]# 

[root@pulp-prod ~]# hostname
pulp-prod


[root@pulp-prod ~]# pulp-admin login -u admin -p admin
Successfully logged in. Session certificate will expire at Jan 16 19:25:42 2015
GMT.

[root@pulp-prod ~]# 
[root@pulp-prod ~]# 
[root@pulp-prod ~]# 
[root@pulp-prod ~]# pulp-admin rpm repo create --repo-id zoo --feed https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/
Successfully created repository [zoo]

[root@pulp-prod ~]# 
[root@pulp-prod ~]# 
[root@pulp-prod ~]# 
[root@pulp-prod ~]# pulp-admin rpm repo sync run --repo-id zoo
+----------------------------------------------------------------------+
                     Synchronizing Repository [zoo]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.


Downloading metadata...
[/]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       32/32 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[-]
... completed


Task Succeeded



Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[==================================================] 100%
32 of 32 items
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[==================================================] 100%
4 of 4 items
... completed

Publishing Comps file
[==================================================] 100%
3 of 3 items
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Publishing files to web
[-]
... completed

Writing Listings File
[-]
... completed


Task Succeeded

Comment 5 Randy Barlow 2015-02-05 21:47:07 UTC
Pulp 2.5.2 has been released.