Bug 754430

Summary: Product promotion fails as katello-jobs doesn't start
Product: [Retired] Katello Reporter: Sachin Ghai <sghai>
Component: Webservice APIAssignee: Lukas Zapletal <lzap>
Status: CLOSED DUPLICATE QA Contact: Katello QA List <katello-qa-list>
Severity: high Docs Contact:
Priority: unspecified    
Version: 1.XCC: lzap
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 754306 Environment:
Last Closed: 2011-11-16 14:35:43 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 755097    

Description Sachin Ghai 2011-11-16 13:17:42 UTC
Description of problem:
I created a repo and synced it and later tried to promote this product to anothe env. but product promotion remains in promoting state.

No backtrace in production.log.



Version-Release number of selected component (if applicable):
[root@dhcp201-152 repos]# rpm -qa | grep katello
katello-configure-0.1.13-1.git.0.522af71.fc15.noarch
katello-cli-common-0.1.12-1.git.0.183b350.fc15.noarch
katello-glue-pulp-0.1.104-1.git.0.26d9620.fc15.x86_64
katello-cli-0.1.12-1.git.0.183b350.fc15.noarch
katello-common-0.1.104-1.git.0.26d9620.fc15.noarch
katello-0.1.104-1.git.0.26d9620.fc15.x86_64
katello-glue-foreman-0.1.104-1.git.0.26d9620.fc15.x86_64
katello-all-0.1.104-1.git.0.26d9620.fc15.x86_64
katello-glue-candlepin-0.1.104-1.git.0.26d9620.fc15.x86_64
katello-repos-0.1.3-1.git.0.db2bd1d.fc15.noarch
[root@dhcp201-152 repos]# rpm -qa | grep pulp
katello-glue-pulp-0.1.104-1.git.0.26d9620.fc15.x86_64
pulp-common-0.0.244-5.fc15.noarch
mod_wsgi-3.2-6.pulp.fc15.x86_64
pulp-0.0.244-5.fc15.noarch
m2crypto-0.21.1.pulp-5.fc15.x86_64
[root@dhcp201-152 repos]# 


How reproducible:
always

Steps to Reproduce:
1.
2.
3.
  
Actual results:
product promotion remain in promoting state for a long time.

Expected results:
product promotion should work

Additional info:

logs from production.log:
======================

Started GET "/katello//changesets/3/promotion_progress/?_=1321445359726" for 10.65.193.48 at Wed Nov 16 17:36:45 +0530 2011
  Processing by ChangesetsController#promotion_progress as JSON
  Parameters: {"id"=>"3", "_"=>"1321445359726"}
Completed 200 OK in 11ms (Views: 0.4ms | ActiveRecord: 2.9ms)


Started GET "/katello//changesets/3/promotion_progress/?_=1321445367777" for 10.65.193.48 at Wed Nov 16 17:36:53 +0530 2011
  Processing by ChangesetsController#promotion_progress as JSON
  Parameters: {"id"=>"3", "_"=>"1321445367777"}
Completed 200 OK in 10ms (Views: 0.3ms | ActiveRecord: 2.7ms)


Started GET "/katello//changesets/3/promotion_progress/?_=1321445375835" for 10.65.193.48 at Wed Nov 16 17:37:01 +0530 2011
  Processing by ChangesetsController#promotion_progress as JSON
  Parameters: {"id"=>"3", "_"=>"1321445375835"}
Completed 200 OK in 10ms (Views: 0.3ms | ActiveRecord: 2.9ms)


Started GET "/katello//changesets/3/promotion_progress/?_=1321445383891" for 10.65.193.48 at Wed Nov 16 17:37:09 +0530 2011
  Processing by ChangesetsController#promotion_progress as JSON
  Parameters: {"id"=>"3", "_"=>"1321445383891"}
Completed 200 OK in 10ms (Views: 0.3ms | ActiveRecord: 2.8ms)

Comment 1 Sachin Ghai 2011-11-16 13:21:20 UTC
Steps to reproduce:
====================

1. Create a product /repo and sync it
2. Create a changeset and add the product to this changeset
3. promote the product.

--
katello> changeset update --name=test --org=ACME_Corporation --env=test --add_product=pulp
Successfully updated changeset [ test ]
katello> changeset promote --name=test --org=ACME_Corporation --env=testkatello> exit
--

Comment 2 Lukas Zapletal 2011-11-16 13:24:57 UTC
Hmmm I provisioned a box now and getting this during katello-configure:

err: /Stage[main]/Katello::Service/Service[katello-jobs]/ensure: change from stopped to running failed: Could not start Service[katello-jobs]: Execution of '/sbin/service katello-jobs start' returned 1:  at /usr/share/katello/install/puppet/modules/katello/manifests/service.pp:10

Taking.

Comment 3 Lukas Zapletal 2011-11-16 14:20:38 UTC
Okay, this is annoying. I provisioned several boxes and from time to time delayed_jobs do not want to start. This is the error:


[root@test ~]# cat /var/log/katello/jobs-startup.log
/usr/lib/ruby/gems/1.8/gems/daemons-1.1.4/lib/daemons/pidfile.rb:94:in `initialize': Permission denied - /usr/share/katello/tmp/pids/delayed_job.pid (Errno::EACCES)
	from /usr/lib/ruby/gems/1.8/gems/daemons-1.1.4/lib/daemons/pidfile.rb:94:in `open'
	from /usr/lib/ruby/gems/1.8/gems/daemons-1.1.4/lib/daemons/pidfile.rb:94:in `pid='
	from /usr/lib/ruby/gems/1.8/gems/daemons-1.1.4/lib/daemons/application.rb:253:in `start_proc'
	from /usr/lib/ruby/gems/1.8/gems/daemons-1.1.4/lib/daemons/application.rb:293:in `start'
	from /usr/lib/ruby/gems/1.8/gems/daemons-1.1.4/lib/daemons/controller.rb:70:in `run'
	from /usr/lib/ruby/gems/1.8/gems/daemons-1.1.4/lib/daemons.rb:195:in `run_proc'
	from /usr/lib/ruby/gems/1.8/gems/daemons-1.1.4/lib/daemons/cmdline.rb:109:in `call'
	from /usr/lib/ruby/gems/1.8/gems/daemons-1.1.4/lib/daemons/cmdline.rb:109:in `catch_exceptions'
	from /usr/lib/ruby/gems/1.8/gems/daemons-1.1.4/lib/daemons.rb:194:in `run_proc'
	from /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/command.rb:81:in `run_process'
	from /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/command.rb:75:in `daemonize'
	from /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/command.rb:73:in `times'
	from /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/command.rb:73:in `daemonize'
	from script/delayed_job:5

The thing is /usr/share/katello/tmp/pids/ has wrong permissions - does not permit "katello" user to write there. As we do not explicitly create this directory it is created by default by the first process that starts. If it's katello-jobs thats fine (owner katello). If it's katello (thin), then this gets root owner because katello is initialized as root (and then it's owner change to katello).

The solution is to distribute this directory in the RPM (with correct permissions).

Comment 4 Lukas Zapletal 2011-11-16 14:33:23 UTC
979a137 754430 - Product promotion fails as katello-jobs doesn't start

Comment 5 Jeff Weiss 2011-11-16 14:35:43 UTC

*** This bug has been marked as a duplicate of bug 754306 ***