Bug 1145723

Summary:	As a user, I would like my Pulp logs to go to logical locations
Product:	[Retired] Pulp	Reporter:	Bryan Kearney <bkearney>
Component:	user-experience	Assignee:	Chris Duryee <cduryee>
Status:	CLOSED UPSTREAM	QA Contact:	Preethi Thomas <pthomas>
Severity:	low	Docs Contact:
Priority:	high
Version:	Master	CC:	bkearney, cduryee, dcleal, katello-bugs, katello-qa-list, mhrivnak, pthomas, rbarlow, stbenjam
Target Milestone:	---	Keywords:	Triaged
Target Release:	2.6.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:	1142113	Environment:
Last Closed:	2015-02-28 22:22:15 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1142113

Comment 1 Michael Hrivnak 2014-09-26 14:28:31 UTC

I have started a discussion on the original bug suggesting we close it.

Comment 2 Randy Barlow 2014-10-14 18:48:27 UTC

The downstream bug corresponding to this issue has a comment saying they would like it if we:

1) Renamed the log files to include the word "startup" in the name (we may or may not be able to do this, not sure about the Celery API),

and,

2) Inject a log message to those files like "pulp started successfully, transitioning to syslog for future logs".

Comment 3 Chris Duryee 2014-10-23 20:20:15 UTC

https://github.com/pulp/pulp/pull/1259

Comment 4 Chris Duryee 2014-11-04 20:16:35 UTC

merged to 2.5-dev and master

Comment 5 Chris Duryee 2014-12-23 20:52:37 UTC

fixed in pulp 2.6.0-0.2.beta

Comment 6 Preethi Thomas 2015-01-14 17:09:51 UTC

Fails-qa
[root@ibm-x3550m3-09 ~]# rpm -qa pulp-server
pulp-server-2.6.0-0.2.beta.el6.noarch
[root@ibm-x3550m3-09 ~
Starting pulp-worker fails with permission error


celery init v10.0.
Using config script: /etc/default/pulp_workers
- Creating default directory: '/var/run/pulp'
- Changing permissions of '/var/run/pulp' to 02755
- Changing owner of '/var/run/pulp' to 'apache'
celery multi v3.1.11 (Cipater)
> Starting nodes...
        > reserved_resource_worker-0.lab.eng.bos.redhat.com: OK
        > reserved_resource_worker-1.lab.eng.bos.redhat.com: Traceback (most recent call last):
  File "/usr/lib64/python2.6/runpy.py", line 122, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code
    exec code in run_globals
  File "/usr/lib/python2.6/site-packages/celery/__main__.py", line 54, in <module>
    main()
  File "/usr/lib/python2.6/site-packages/celery/__main__.py", line 30, in main
    main()
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 81, in main
    cmd.execute_from_commandline(argv)
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 769, in execute_from_commandline
    super(CeleryCommand, self).execute_from_commandline(argv)))
  File "/usr/lib/python2.6/site-packages/celery/bin/base.py", line 306, in execute_from_commandline
    return self.handle_argv(self.prog_name, argv[1:])
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 761, in handle_argv
    return self.execute(command, argv)
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 693, in execute
    ).run_from_argv(self.prog_name, argv[1:], command=argv[0])
  File "/usr/lib/python2.6/site-packages/celery/bin/worker.py", line 178, in run_from_argv
    self.maybe_detach([command] + sys.argv[1:])
  File "/usr/lib/python2.6/site-packages/celery/bin/worker.py", line 185, in maybe_detach
    detached_celeryd(self.app).execute_from_commandline(argv)
  File "/usr/lib/python2.6/site-packages/celery/bin/celeryd_detach.py", line 160, in execute_from_commandline
    **vars(options)
  File "/usr/lib/python2.6/site-packages/celery/bin/celeryd_detach.py", line 42, in detach
    with detached(logfile, pidfile, uid, gid, umask, working_directory, fake):
  File "/usr/lib/python2.6/site-packages/celery/platforms.py", line 318, in open
    self.after_chdir()
  File "/usr/lib/python2.6/site-packages/celery/platforms.py", line 388, in after_chdir_do
    logfile and open(logfile, 'a').close()
IOError: [Errno 13] Permission denied: '/var/log/pulp/reserved_resource_worker-0.log'
OK
        > reserved_resource_worker-2.lab.eng.bos.redhat.com: Traceback (most recent call last):
  File "/usr/lib64/python2.6/runpy.py", line 122, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code
    exec code in run_globals
  File "/usr/lib/python2.6/site-packages/celery/__main__.py", line 54, in <module>
    main()
  File "/usr/lib/python2.6/site-packages/celery/__main__.py", line 30, in main
    main()
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 81, in main
    cmd.execute_from_commandline(argv)
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 769, in execute_from_commandline
    super(CeleryCommand, self).execute_from_commandline(argv)))
  File "/usr/lib/python2.6/site-packages/celery/bin/base.py", line 306, in execute_from_commandline
    return self.handle_argv(self.prog_name, argv[1:])
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 761, in handle_argv
    return self.execute(command, argv)
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 693, in execute
    ).run_from_argv(self.prog_name, argv[1:], command=argv[0])
  File "/usr/lib/python2.6/site-packages/celery/bin/worker.py", line 178, in run_from_argv
    self.maybe_detach([command] + sys.argv[1:])
  File "/usr/lib/python2.6/site-packages/celery/bin/worker.py", line 185, in maybe_detach
    detached_celeryd(self.app).execute_from_commandline(argv)
  File "/usr/lib/python2.6/site-packages/celery/bin/celeryd_detach.py", line 160, in execute_from_commandline
    **vars(options)
  File "/usr/lib/python2.6/site-packages/celery/bin/celeryd_detach.py", line 42, in detach
    with detached(logfile, pidfile, uid, gid, umask, working_directory, fake):
  File "/usr/lib/python2.6/site-packages/celery/platforms.py", line 318, in open
    self.after_chdir()
  File "/usr/lib/python2.6/site-packages/celery/platforms.py", line 388, in after_chdir_do
    logfile and open(logfile, 'a').close()
IOError: [Errno 13] Permission denied: '/var/log/pulp/reserved_resource_worker-1.log'
OK
        > reserved_resource_worker-3.lab.eng.bos.redhat.com: Traceback (most recent call last):
  File "/usr/lib64/python2.6/runpy.py", line 122, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code
    exec code in run_globals
  File "/usr/lib/python2.6/site-packages/celery/__main__.py", line 54, in <module>
    main()
  File "/usr/lib/python2.6/site-packages/celery/__main__.py", line 30, in main
    main()
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 81, in main
    cmd.execute_from_commandline(argv)
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 769, in execute_from_commandline
    super(CeleryCommand, self).execute_from_commandline(argv)))
  File "/usr/lib/python2.6/site-packages/celery/bin/base.py", line 306, in execute_from_commandline
    return self.handle_argv(self.prog_name, argv[1:])
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 761, in handle_argv
    return self.execute(command, argv)
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 693, in execute
    ).run_from_argv(self.prog_name, argv[1:], command=argv[0])
  File "/usr/lib/python2.6/site-packages/celery/bin/worker.py", line 178, in run_from_argv
    self.maybe_detach([command] + sys.argv[1:])
  File "/usr/lib/python2.6/site-packages/celery/bin/worker.py", line 185, in maybe_detach
    detached_celeryd(self.app).execute_from_commandline(argv)
  File "/usr/lib/python2.6/site-packages/celery/bin/celeryd_detach.py", line 160, in execute_from_commandline
    **vars(options)
  File "/usr/lib/python2.6/site-packages/celery/bin/celeryd_detach.py", line 42, in detach
    with detached(logfile, pidfile, uid, gid, umask, working_directory, fake):
  File "/usr/lib/python2.6/site-packages/celery/platforms.py", line 318, in open
    self.after_chdir()
  File "/usr/lib/python2.6/site-packages/celery/platforms.py", line 388, in after_chdir_do
    logfile and open(logfile, 'a').close()
IOError: [Errno 13] Permission denied: '/var/log/pulp/reserved_resource_worker-2.log'
OK
        > reserved_resource_worker-4.lab.eng.bos.redhat.com: Traceback (most recent call last):
  File "/usr/lib64/python2.6/runpy.py", line 122, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code
    exec code in run_globals
  File "/usr/lib/python2.6/site-packages/celery/__main__.py", line 54, in <module>
    main()
  File "/usr/lib/python2.6/site-packages/celery/__main__.py", line 30, in main
    main()
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 81, in main
    cmd.execute_from_commandline(argv)
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 769, in execute_from_commandline
    super(CeleryCommand, self).execute_from_commandline(argv)))
  File "/usr/lib/python2.6/site-packages/celery/bin/base.py", line 306, in execute_from_commandline
    return self.handle_argv(self.prog_name, argv[1:])
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 761, in handle_argv
    return self.execute(command, argv)
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 693, in execute
    ).run_from_argv(self.prog_name, argv[1:], command=argv[0])
  File "/usr/lib/python2.6/site-packages/celery/bin/worker.py", line 178, in run_from_argv
    self.maybe_detach([command] + sys.argv[1:])
  File "/usr/lib/python2.6/site-packages/celery/bin/worker.py", line 185, in maybe_detach
    detached_celeryd(self.app).execute_from_commandline(argv)
  File "/usr/lib/python2.6/site-packages/celery/bin/celeryd_detach.py", line 160, in execute_from_commandline
    **vars(options)
  File "/usr/lib/python2.6/site-packages/celery/bin/celeryd_detach.py", line 42, in detach
    with detached(logfile, pidfile, uid, gid, umask, working_directory, fake):
  File "/usr/lib/python2.6/site-packages/celery/platforms.py", line 318, in open
    self.after_chdir()
  File "/usr/lib/python2.6/site-packages/celery/platforms.py", line 388, in after_chdir_do
    logfile and open(logfile, 'a').close()
IOError: [Errno 13] Permission denied: '/var/log/pulp/reserved_resource_worker-3.log'
OK
        > reserved_resource_worker-5.lab.eng.bos.redhat.com: Traceback (most recent call last):
  File "/usr/lib64/python2.6/runpy.py", line 122, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code
    exec code in run_globals
  File "/usr/lib/python2.6/site-packages/celery/__main__.py", line 54, in <module>
    main()
  File "/usr/lib/python2.6/site-packages/celery/__main__.py", line 30, in main
    main()
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 81, in main
    cmd.execute_from_commandline(argv)
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 769, in execute_from_commandline
    super(CeleryCommand, self).execute_from_commandline(argv)))
  File "/usr/lib/python2.6/site-packages/celery/bin/base.py", line 306, in execute_from_commandline
    return self.handle_argv(self.prog_name, argv[1:])
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 761, in handle_argv
    return self.execute(command, argv)
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 693, in execute
    ).run_from_argv(self.prog_name, argv[1:], command=argv[0])
  File "/usr/lib/python2.6/site-packages/celery/bin/worker.py", line 178, in run_from_argv
    self.maybe_detach([command] + sys.argv[1:])
  File "/usr/lib/python2.6/site-packages/celery/bin/worker.py", line 185, in maybe_detach
    detached_celeryd(self.app).execute_from_commandline(argv)
  File "/usr/lib/python2.6/site-packages/celery/bin/celeryd_detach.py", line 160, in execute_from_commandline
    **vars(options)
  File "/usr/lib/python2.6/site-packages/celery/bin/celeryd_detach.py", line 42, in detach
    with detached(logfile, pidfile, uid, gid, umask, working_directory, fake):
  File "/usr/lib/python2.6/site-packages/celery/platforms.py", line 318, in open
    self.after_chdir()
  File "/usr/lib/python2.6/site-packages/celery/platforms.py", line 388, in after_chdir_do
    logfile and open(logfile, 'a').close()
IOError: [Errno 13] Permission denied: '/var/log/pulp/reserved_resource_worker-4.log'
OK
        > reserved_resource_worker-6.lab.eng.bos.redhat.com: Traceback (most recent call last):
  File "/usr/lib64/python2.6/runpy.py", line 122, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code
    exec code in run_globals
  File "/usr/lib/python2.6/site-packages/celery/__main__.py", line 54, in <module>
    main()
  File "/usr/lib/python2.6/site-packages/celery/__main__.py", line 30, in main
    main()
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 81, in main
    cmd.execute_from_commandline(argv)
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 769, in execute_from_commandline
    super(CeleryCommand, self).execute_from_commandline(argv)))
  File "/usr/lib/python2.6/site-packages/celery/bin/base.py", line 306, in execute_from_commandline
    return self.handle_argv(self.prog_name, argv[1:])
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 761, in handle_argv
    return self.execute(command, argv)
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 693, in execute
    ).run_from_argv(self.prog_name, argv[1:], command=argv[0])
  File "/usr/lib/python2.6/site-packages/celery/bin/worker.py", line 178, in run_from_argv
    self.maybe_detach([command] + sys.argv[1:])
  File "/usr/lib/python2.6/site-packages/celery/bin/worker.py", line 185, in maybe_detach
    detached_celeryd(self.app).execute_from_commandline(argv)
  File "/usr/lib/python2.6/site-packages/celery/bin/celeryd_detach.py", line 160, in execute_from_commandline
    **vars(options)
  File "/usr/lib/python2.6/site-packages/celery/bin/celeryd_detach.py", line 42, in detach
    with detached(logfile, pidfile, uid, gid, umask, working_directory, fake):
  File "/usr/lib/python2.6/site-packages/celery/platforms.py", line 318, in open
    self.after_chdir()
  File "/usr/lib/python2.6/site-packages/celery/platforms.py", line 388, in after_chdir_do
    logfile and open(logfile, 'a').close()
IOError: [Errno 13] Permission denied: '/var/log/pulp/reserved_resource_worker-5.log'
OK
        > reserved_resource_worker-7.lab.eng.bos.redhat.com: Traceback (most recent call last):
  File "/usr/lib64/python2.6/runpy.py", line 122, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code
    exec code in run_globals
  File "/usr/lib/python2.6/site-packages/celery/__main__.py", line 54, in <module>
    main()
  File "/usr/lib/python2.6/site-packages/celery/__main__.py", line 30, in main
    main()
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 81, in main
    cmd.execute_from_commandline(argv)
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 769, in execute_from_commandline
    super(CeleryCommand, self).execute_from_commandline(argv)))
  File "/usr/lib/python2.6/site-packages/celery/bin/base.py", line 306, in execute_from_commandline
    return self.handle_argv(self.prog_name, argv[1:])
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 761, in handle_argv
    return self.execute(command, argv)
  File "/usr/lib/python2.6/site-packages/celery/bin/celery.py", line 693, in execute
    ).run_from_argv(self.prog_name, argv[1:], command=argv[0])
  File "/usr/lib/python2.6/site-packages/celery/bin/worker.py", line 178, in run_from_argv
    self.maybe_detach([command] + sys.argv[1:])
  File "/usr/lib/python2.6/site-packages/celery/bin/worker.py", line 185, in maybe_detach
    detached_celeryd(self.app).execute_from_commandline(argv)
  File "/usr/lib/python2.6/site-packages/celery/bin/celeryd_detach.py", line 160, in execute_from_commandline
    **vars(options)
  File "/usr/lib/python2.6/site-packages/celery/bin/celeryd_detach.py", line 42, in detach
    with detached(logfile, pidfile, uid, gid, umask, working_directory, fake):
  File "/usr/lib/python2.6/site-packages/celery/platforms.py", line 318, in open
    self.after_chdir()
  File "/usr/lib/python2.6/site-packages/celery/platforms.py", line 388, in after_chdir_do
    logfile and open(logfile, 'a').close()
IOError: [Errno 13] Permission denied: '/var/log/pulp/reserved_resource_worker-6.log'
OK
[root@qe-blade-03 ~]#
 

[root@qe-blade-03 ~]# ls -la /var/log/pulp
total 40
drwxr-xr-x.  2 apache apache 4096 Jan 14 08:09 .
drwxr-xr-x. 14 root   root   4096 Jan 14 08:05 ..
-rw-r--r--.  1 root   root    308 Jan 14 08:09 reserved_resource_worker-0.log
-rw-r--r--.  1 root   root    308 Jan 14 08:09 reserved_resource_worker-1.log
-rw-r--r--.  1 root   root    308 Jan 14 08:09 reserved_resource_worker-2.log
-rw-r--r--.  1 root   root    308 Jan 14 08:09 reserved_resource_worker-3.log
-rw-r--r--.  1 root   root    308 Jan 14 08:09 reserved_resource_worker-4.log
-rw-r--r--.  1 root   root    308 Jan 14 08:09 reserved_resource_worker-5.log
-rw-r--r--.  1 root   root    308 Jan 14 08:09 reserved_resource_worker-6.log
-rw-r--r--.  1 apache root    308 Jan 14 08:09 reserved_resource_worker-7.log

Deleting the log files and restarting the pulp-workers shows this



2015-01-14 08:09:43 ********************************************************
2015-01-14 08:09:43 * Celery startup requested. After startup is complete, *
2015-01-14 08:09:43 * messages will be logged to /var/log/messages.        *
2015-01-14 08:09:43 ****

Comment 7 Chris Duryee 2015-01-15 21:19:32 UTC

https://github.com/pulp/pulp/pull/1529

Comment 8 Chris Duryee 2015-01-16 23:00:02 UTC

pulp 2.6.0 beta 5

Comment 9 Preethi Thomas 2015-01-30 20:19:41 UTC

verified on rhel6 & rhel7

[root@cloud-qe-4 ~]# rpm -qa pulprpm -qa pulp-server
pulp-server-2.6.0-0.5.beta.el7.noarch

Comment 10 Brian Bouterse 2015-02-28 22:22:15 UTC

Moved to https://pulp.plan.io/issues/542