Bug 973333 - various services doesn't output error to log when fails on parsing conf
Summary: various services doesn't output error to log when fails on parsing conf
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: distribution
Version: 3.0
Hardware: x86_64
OS: Linux
medium
high
Target Milestone: ---
: 5.0 (RHEL 7)
Assignee: Alan Pevec
QA Contact: Jaroslav Henner
URL:
Whiteboard:
: 972359 973328 (view as bug list)
Depends On: 972359
Blocks: 973328
TreeView+ depends on / blocked
 
Reported: 2013-06-11 17:16 UTC by Jaroslav Henner
Modified: 2016-04-26 15:29 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Known Issue
Doc Text:
If a configuration file contains invalid or otherwise unparseable data OpenStack services that attempt to read values from it will fail to start. While this is expected behaviour in this situation the service initialization scripts do not currently display any output indicating that there was a failure. To work around this issue when changing configuration files restart the service as normal and then manually confirm that the service is running. For example: # service openstack-keystone restart Stopping keystone: [ OK ] Starting keystone: [ OK ] # service openstack-keystone status keystone (pid 12632) is running...
Clone Of: 972359
Environment:
Last Closed: 2015-12-15 20:48:40 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
openstack-nova-api.patch (305 bytes, patch)
2014-03-26 13:59 UTC, Jaroslav Henner
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1190001 0 None None None Never

Description Jaroslav Henner 2013-06-11 17:16:50 UTC
What I found for quantum holds for keystone and nova. My twisted logic says "what holds for some, surely holds for all"... so I am filling this general bug.


+++ This bug was initially created as a clone of Bug #972359 +++

Description of problem:
quantum fails and there is nothing in the log

if there is 
[SECURITYGROUP]
# Firewall driver for realizing quantum security group function
quantum.agent.linux.iptables_firewall.IptablesFirewallDriver

in the /etc/quantum/plugin.ini, then quantum doesn't start and doesn't output anything to server.log


Version-Release number of selected component (if applicable):
openstack-quantum-2013.1.1-10.el6ost.noarch
openstack-quantum-linuxbridge-2013.1.1-10.el6ost.noarch


How reproducible:
always


Steps to Reproduce:
1. change the config accordingly to description
2. restart quantum-server
3.

Actual results:
quantum is dead, nothing in log.

Expected results:
error in log

Additional info:

--- Additional comment from Ryan O'Hara on 2013-06-11 02:33:32 CEST ---

How did you install? Can you attach the plugin.ini file?

--- Additional comment from Jaroslav Henner on 2013-06-11 14:22:19 CEST ---

(In reply to Ryan O'Hara from comment #1)
> How did you install? Can you attach the plugin.ini file?

With broken Packstack (see the referred clone of this bug), but you can simulate it without the packstack, with the RPMs for sure. Obviously, the bug won't appear with Devstack because Devstack IMHO doesn't use the init scripts.

I don't have that config, but you should be able to reproduce it:

[root@master-02 ~]# echo 'I am evil as hell.' >> /etc/quantum/plugin.ini 
[root@master-02 ~]# tail -fn0 /var/log/quantum/linuxbridge-agent.log &
[2] 16976
[root@master-02 ~]# /etc/init.d/quantum-linuxbridge-agent restart
Stopping quantum-linuxbridge-agent:                        [  OK  ]
Starting quantum-linuxbridge-agent:                        [  OK  ]
[root@master-02 ~]# /etc/init.d/quantum-linuxbridge-agent status
quantum-linuxbridge-agent dead but pid file exists
[root@master-02 ~]# kill %
[root@master-02 ~]# 

See? No message. Only death.

Comment 1 Jaroslav Henner 2013-06-11 17:19:40 UTC
*** Bug 973328 has been marked as a duplicate of this bug. ***

Comment 2 Jaroslav Henner 2013-06-11 17:19:51 UTC
*** Bug 972359 has been marked as a duplicate of this bug. ***

Comment 7 Jaroslav Henner 2013-06-13 13:10:02 UTC
Note it is possible to debug the invalid conf if one knows does the init spawn the service process:

[root@folsom-rhel6 ~]# echo 'Evil line' >> /etc/keystone/keystone.conf
[root@folsom-rhel6 ~]# /usr/bin/python /usr/bin/keystone-all --config-file 
/usr/share/keystone/keystone-dist.conf --config-file /etc/keystone/keystone.conf
Traceback (most recent call last):
  File "/usr/bin/keystone-all", line 82, in <module>
    default_config_files=config_files)
  File "/usr/lib/python2.6/site-packages/oslo/config/cfg.py", line 1182, in __call__
    self._parse_config_files()
  File "/usr/lib/python2.6/site-packages/oslo/config/cfg.py", line 1653, in _parse_config_files
    raise ConfigFileParseError(pe.filename, str(pe))
oslo.config.cfg.ConfigFileParseError: Failed to parse /etc/keystone/keystone.conf: at /etc/keystone/keystone.conf:227, No ':' or '=' found in assignment: 'Evil line'

Comment 8 Jaroslav Henner 2013-06-16 20:31:27 UTC
This bug appears also when there are problems with permission when reading the config files. Rising severity because this is really nasty. I had to debug with strace -f to see why my quantum-linuxbridge-agent doesn't start

Comment 10 Jaroslav Henner 2014-03-26 13:59:06 UTC
Created attachment 879008 [details]
openstack-nova-api.patch

This is not a issue on upstream:

https://bugs.launchpad.net/oslo/+bug/1190001

It is either, depending on point of view:
 * a bug in the init scripts, or it is 
 * the config parsing is not reporting to the log because the logging is initialized after the config parsing (which makes sense because you can configure logging there).

Check how the init script behaves when it is not dumping the outputs to a sewage (apply the attached patch).

[root@jhenner-node-permanent-1 ~]# /etc/init.d/openstack-nova-api restart
Stopping openstack-nova-api:                               [FAILED]
Starting openstack-nova-api:                               [  OK  ]
[root@jhenner-node-permanent-1 ~]# /etc/init.d/openstack-nova-api.modified restart
Stopping openstack-nova-api:                               [FAILED]
Starting openstack-nova-api: Traceback (most recent call last):
  File "/usr/bin/nova-api", line 10, in <module>
    sys.exit(main())
  File "/usr/lib/python2.6/site-packages/nova/cmd/api.py", line 40, in main
    config.parse_args(sys.argv)
  File "/usr/lib/python2.6/site-packages/nova/config.py", line 37, in parse_args
    default_config_files=default_config_files)
  File "/usr/lib/python2.6/site-packages/oslo/config/cfg.py", line 1636, in __call__
    else sys.argv[1:])
  File "/usr/lib/python2.6/site-packages/oslo/config/cfg.py", line 2137, in _parse_cli_opts
    return self._parse_config_files()
  File "/usr/lib/python2.6/site-packages/oslo/config/cfg.py", line 2151, in _parse_config_files
    ConfigParser._parse_file(config_file, namespace)
  File "/usr/lib/python2.6/site-packages/oslo/config/cfg.py", line 1256, in _parse_file
    raise ConfigFileParseError(pe.filename, str(pe))
oslo.config.cfg.ConfigFileParseError: Failed to parse /etc/nova/nova.conf: at /etc/nova/nova.conf:3494, No ':' or '=' found in assignment: 'I am evil as hell.'

Comment 11 Alan Pevec 2014-05-16 14:21:58 UTC
>  * a bug in the init scripts, or it is 

This should be fixed as a part of bug 1036515 where startup messages are redirected to the separate startuplog /var/log/$prog/$prog-startup.log

Comment 12 Alan Pevec 2014-05-16 14:25:38 UTC
comment 11 applies for el6
For el7 systemd services output is collected by the systemd journal.


Note You need to log in before you can comment on or make changes to this bug.