Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1090821

Summary: mv horizon.log while running will cause UI 500 internal server error
Product: Red Hat OpenStack Reporter: Tzach Shefi <tshefi>
Component: python-django-horizonAssignee: Matthias Runge <mrunge>
Status: CLOSED NOTABUG QA Contact: Ami Jeain <ajeain>
Severity: high Docs Contact:
Priority: unspecified    
Version: 5.0 (RHEL 7)CC: aortega, jpichon, mrunge, tshefi, yeylon
Target Milestone: ---Keywords: Triaged
Target Release: 5.0 (RHEL 7)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-06-10 12:19:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
audit.log
none
Horizon.log
none
audit.log none

Description Tzach Shefi 2014-04-24 08:53:07 UTC
Description of problem:  moving horizon.log while system is up, causes 500 internal server error on UI. 

Version-Release number of selected component (if applicable):
RHEL7
openstack-dashboard-2014.1-1.el7.noarch
python-django-openstack-auth-1.1.5-1.el7.noarch


How reproducible:
Not sure first time happens to me.

Steps to Reproduce:
1. Setup system, enable Horizon logging
2. while system is running # mv horizon.log horizon.log2  
3. restart httpd 
4. Try to access UI -> 500 internal server error 

I now know I should have stopped services before moving log, but I didn't and this could happen to others as well. 

Actual results:
New horizon log created but with root:root 
UI fails with 500 internal error

Expected results:
I'd expect new log to be created correctly and UI to work.

Additional info:
Changing log apache:apache didn't resolve, full system reboot didn't resolve either. 

The only workaround I found was to set Horizon's debug to false restart httpd and UI worked. But when I tired debug=true again fails with same 500 error, moving log file broke something.

Comment 1 Matthias Runge 2014-04-27 08:58:29 UTC
please add ls -lZ /var/log/horizon/horizon.log
I suspect a SELinux issue here and this is probably a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1089371

Comment 2 Tzach Shefi 2014-04-30 09:02:45 UTC
Original setup reprovisioned sorry for delay.
 
New setup same problem when I tired to recreate this bug, noticed I also did openstack-service restart, not just httpd. 
Forgot to mention this on original bug, resting only httpd doesn't recreate problem.

Below output of requested command before moving log.
[root@opens-vdsb horizon(keystone_admin)]# ls -lZ
-rw-r--r--. apache apache system_u:object_r:httpd_log_t:s0 horizon.log


And after moving log, dashbaord -> Internal Server Error
[root@opens-vdsb horizon(keystone_admin)]# ls -lZ
-rw-r--r--. apache apache system_u:object_r:httpd_log_t:s0 horizon.log
-rw-r--r--. apache apache system_u:object_r:httpd_log_t:s0 horizon.log2

Comment 3 Tzach Shefi 2014-05-14 12:40:12 UTC
Also happens on RHOS 5 beta
RHEL7
openstack-dashboard-theme-2014.1-4.el7ost.noarch
openstack-dashboard-2014.1-4.el7ost.noarch

Comment 5 Matthias Runge 2014-05-16 17:51:22 UTC
I would argue, this is not an horizon, but an httpd bug (if at all).

To my verification, could you please attach the corresponding snippet of audit.log after removing horizon log?

Comment 6 Tzach Shefi 2014-05-19 07:10:01 UTC
Created attachment 897026 [details]
audit.log

Comment 7 Tzach Shefi 2014-05-19 07:14:37 UTC
Note uploaded audit log enabled after system was broke, it doesn't show much. On my next build I'll enable audit.log before moving horizon's log, might be more helpful.

As per it being a Horizon or httpd bug, I can't argue your the expert. My only concern is the dead dashboard, which IMHO shouldn't happen or be handled better.

Comment 8 Matthias Runge 2014-05-19 09:29:32 UTC
agreed. That log does not help at all. It would be more helpful to do:
echo "" > /var/log/audit/audit.log
rm /var/log/horizon/horizon.log
service httpd restart

and then provide that log.

Comment 9 Tzach Shefi 2014-06-09 12:49:58 UTC
We know selinux on rhel7 causes problems, my default procedure is to disables selinux before running packstack (until SE is fixed). 

I have two deployments running, enabling selinux (enforcing) on them causes dashboard to fail, on one with 500 internal server error, on the other "something went wrong" that's before I even touched horizon log, can't recreate this. 

However when I set to permissive mode I was able to access dashboard, best I could do to recreate, deleted horizon.log after logging in to dashabord I get:

Forbidden (403)
CSRF verification failed. Request aborted.


Attached audit.log and Horizon.log. 

Does this help info help? 
Feel free to mess with system if you wish @ 10.35.162.22 usual admin password.

Comment 10 Tzach Shefi 2014-06-09 12:50:35 UTC
Created attachment 904652 [details]
Horizon.log

Comment 11 Tzach Shefi 2014-06-09 12:51:04 UTC
Created attachment 904654 [details]
audit.log

Comment 12 Matthias Runge 2014-06-10 12:19:56 UTC
after conversation on irc, we decided to close this, as it was not reproducable anymore; assuming, this was originally due a selinux denial.