Bug 886709

In most of the cases logs will be collected.

If severe (protocol violation, communication lost) error is detected, then collection is not available, and log file is available at host machine.

On on hand it looks like an improvements, since now all the logs are being collected in a central location (except for the host logs collected by the log collector tool)

However the collection instructions become more complex to the end user or GSS.

Before this change is was always: 
1. Go to host:/tmp/xxx.log
Now it's 
1. Go to engine-host:/var/log/ovirt-engine/host-deploy and if it's not there then,
2. Go to host:/tmp/xxx.log

Is that correct?

Unless something terrible wrong logs will be at engine. Much easier.

Also, in fedora and others, /tmp/ is cleared after boot... so you don't have anything.

During deployment you get a message were logs is at host and where they are stored at engine. So you know where to look.

Example of events of new implementation:

---

2012-Dec-13, 20:13 Host test1 installed
2012-Dec-13, 20:13 Installing Host test1. Stage: Termination.
2012-Dec-13, 20:13 Installing Host test1. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20121213201308-10.35.0.73.log'.
2012-Dec-13, 20:13 Installing Host test1. Reboot scheduled.	
2012-Dec-13, 20:13 Installing Host test1. Stage: Pre-termination.
2012-Dec-13, 20:13 Installing Host test1. Reboot scheduled.
2012-Dec-13, 20:13 Installing Host test1. Stage: Closing up.
2012-Dec-13, 20:13 Installing Host test1. Stage: Transaction commit.
2012-Dec-13, 20:13 Installing Host test1. Enrolling certificate.
2012-Dec-13, 20:13 Installing Host test1. Setting time.
2012-Dec-13, 20:13 Installing Host test1. Stage: Misc configuration.
2012-Dec-13, 20:13 Installing Host test1. Stage: Package installation.
2012-Dec-13, 20:13 Installing Host test1. Stage: Transaction setup.
2012-Dec-13, 20:13 Installing Host test1. Hardware supports virtualization.
2012-Dec-13, 20:13 Installing Host test1. Stage: Setup validation.
2012-Dec-13, 20:13 Installing Host test1. Enfocing host reboot.
2012-Dec-13, 20:13 Installing Host test1. Logs at host located at: '/tmp/ovirt-host-deploy-20121213201256.log'.
2012-Dec-13, 20:13 Installing Host test1. Stage: Environment customization.
2012-Dec-13, 20:13 Installing Host test1. Stage: Programs detection.
2012-Dec-13, 20:12 Installing Host test1. Stage: Installation packages setup.
2012-Dec-13, 20:12 Installing Host test1. Stage: Environment setup.
2012-Dec-13, 20:12 Installing Host test1. Stage: Initializing.
2012-Dec-13, 20:12 Installing Host test1. Connected to host 10.35.0.73 with SSH key fingerprint: 74:b2:98:dd:b4:31:e3:ef:48:71:90:aa:71:d4:f1:db.

(In reply to comment #3)

> 
> During deployment you get a message were logs is at host and where they are
> stored at engine. So you know where to look.
> 
> 2012-Dec-13, 20:13 Host test1 installed
> 2012-Dec-13, 20:13 Installing Host test1. Stage: Termination.
> 2012-Dec-13, 20:13 Installing Host test1. Retrieving installation logs to:
> '/var/log/ovirt-engine/host-deploy/ovirt-20121213201308-10.35.0.73.log'.

Good

> Also, in fedora and others, /tmp/ is cleared after boot... so you don't have anything.

2012-Dec-13, 20:13 Installing Host test1. Logs at host located at: '/tmp/ovirt-host-deploy-20121213201256.log'. 

Not good, since many times if you can't retrieve the logs then you may have to reboot the machine to retrieve network connection (This is the first thing a remote user (without remote console) will do before actually going to the server room to get console access) 

If /tmp is deleted we may need to consider a different location like /var/log

(In reply to comment #4)
> (In reply to comment #3)

> 2012-Dec-13, 20:13 Installing Host test1. Logs at host located at:
> '/tmp/ovirt-host-deploy-20121213201256.log'. 
> 
> Not good, since many times if you can't retrieve the logs then you may have
> to reboot the machine to retrieve network connection (This is the first
> thing a remote user (without remote console) will do before actually going
> to the server room to get console access) 
> 
> If /tmp is deleted we may need to consider a different location like /var/log

/var/log is managed directory, you cannot put there logs of junk.

managed directory = each file has owner (package).
managed directory = each file is monitored and cleaned up (logrotate and similar).

The deployment process is not installed on the machine, hence should not write to /var/log.

/tmp is the right place, as done in the past.

sysadmin can override this location using /etc/ovirt-host-config.conf, if indeed there is a problem during deployment, host reboot but engine cannot fetch log.

(In reply to comment #4)
> If /tmp is deleted we may need to consider a different location like /var/log

Note that in Fedora 18 after much back and forth the default will be for /tmp to be a tmpfs filesystem. That means it is wiped on reboot. I would expect this to impact upstream ovirt and potentially future versions of RHEV (if this change ever makes it into RHEL).

(In reply to comment #6)
> (In reply to comment #4)
> > If /tmp is deleted we may need to consider a different location like /var/log
> 
> Note that in Fedora 18 after much back and forth the default will be for
> /tmp to be a tmpfs filesystem. That means it is wiped on reboot. I would
> expect this to impact upstream ovirt and potentially future versions of RHEV
> (if this change ever makes it into RHEL).

Known issue, refer to previous comments.

I am not sure it is tmpfs, but sure it is cleared on startup, many distributions already took this approach which is correct.

Thanks!

Right, but the proposed resolution to the scenario Simon put forward looks like being to push a very unintuitive procedure off to documentation (otherwise how would the admin know to tweak the setting in /etc/ovirt-host-config.conf)?

What is the reason for preferring /tmp here over /var/tmp (at least for upstream) which is persisted?

(In reply to comment #8)
> Right, but the proposed resolution to the scenario Simon put forward looks
> like being to push a very unintuitive procedure off to documentation
> (otherwise how would the admin know to tweak the setting in
> /etc/ovirt-host-config.conf)?

The documentation should state that the logs are collected to the engine machine at /var/log/ovirt-engine/host-deploy.

In cause of severe installation failure, the engine will not be able to collect the logs, and the host will not reboot (as there is installation failure), in this case there is a fallback to consult logs at /tmp of the host machine.

In case there is a severe system breakage, that for some reason installation fails and host reboots (I cannot think of a valid sequence), there is another fallback for GSS to extract the logs to a different location by altering a configuration file.

> What is the reason for preferring /tmp here over /var/tmp (at least for
> upstream) which is persisted?

/var/tmp should also be cleared at startup, there should be no real difference between /tmp and /var/tmp except that /tmp is available at boot, while /var/tmp can be  mounted at startup. Both can be cleared at any time, and should be cleared at startup.

(In reply to comment #9)
> (In reply to comment #8)
> > Right, but the proposed resolution to the scenario Simon put forward looks
> > like being to push a very unintuitive procedure off to documentation
> > (otherwise how would the admin know to tweak the setting in
> > /etc/ovirt-host-config.conf)?
> 
> The documentation should state that the logs are collected to the engine
> machine at /var/log/ovirt-engine/host-deploy.
> 
> In cause of severe installation failure, the engine will not be able to
> collect the logs, and the host will not reboot (as there is installation
> failure), in this case there is a fallback to consult logs at /tmp of the
> host machine.
> 
> In case there is a severe system breakage, that for some reason installation
> fails and host reboots (I cannot think of a valid sequence), there is
> another fallback for GSS to extract the logs to a different location by
> altering a configuration file.

OK.

> > What is the reason for preferring /tmp here over /var/tmp (at least for
> > upstream) which is persisted?
> 
> /var/tmp should also be cleared at startup, there should be no real
> difference between /tmp and /var/tmp except that /tmp is available at boot,
> while /var/tmp can be  mounted at startup. Both can be cleared at any time,
> and should be cleared at startup.

The specific difference between them in the FHS is actually that /var/tmp is *not* to be cleared at startup.

(In reply to comment #10)
> The specific difference between them in the FHS is actually that /var/tmp is
> *not* to be cleared at startup.

I have no problem to move this to /var/tmp, same-same. will do.

(In reply to comment #11)
> (In reply to comment #10)
> > The specific difference between them in the FHS is actually that /var/tmp is
> > *not* to be cleared at startup.
> 
> I have no problem to move this to /var/tmp, same-same. will do.

Question, is it wise to break the default python implementation of tmpdir search[1]?

[1] http://docs.python.org/2/library/tempfile.html#tempfile.gettempdir

(In reply to comment #12)
> (In reply to comment #11)
> > (In reply to comment #10)
> > > The specific difference between them in the FHS is actually that /var/tmp is
> > > *not* to be cleared at startup.
> > 
> > I have no problem to move this to /var/tmp, same-same. will do.
> 
> Question, is it wise to break the default python implementation of tmpdir
> search[1]?
> 
> [1] http://docs.python.org/2/library/tempfile.html#tempfile.gettempdir

Good question, I don't have an answer for that.

(In reply to comment #13)
> (In reply to comment #12)
> > (In reply to comment #11)
> > > (In reply to comment #10)
> > > > The specific difference between them in the FHS is actually that /var/tmp is
> > > > *not* to be cleared at startup.
> > > 
> > > I have no problem to move this to /var/tmp, same-same. will do.
> > 
> > Question, is it wise to break the default python implementation of tmpdir
> > search[1]?
> > 
> > [1] http://docs.python.org/2/library/tempfile.html#tempfile.gettempdir
> 
> Good question, I don't have an answer for that.

So I leave this as-is for now. Thanks!

(In reply to comment #14)
> So I leave this as-is for now. Thanks!

Ack, based on previous answers I will clone to docs to ensure that the logging locations (both host and engine) are listed in the appropriate appendix but will leave out the procedure for handling the unlikely/unknown scenario where the host install fails and it reboots without logging to the engine.

Verified sf4

(In reply to comment #2)
> Before this change is was always: 
> 1. Go to host:/tmp/xxx.log
> Now it's 
> 1. Go to engine-host:/var/log/ovirt-engine/host-deploy and if it's not there
> then,
> 2. Go to host:/tmp/xxx.log
> 
> Is that correct?

Hey Alon,

I'm trying to resolve BZ#889333, and update "Red Hat Enterprise Virtualization Host Log Files"

https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_Virtualization/3.1/html-single/Installation_Guide/index.html#Red_Hat_Enterprise_Virtualization_Host_Log_Files1

Host side installation logs go to /tmp/xxx.log. What is the value of "xxx"?

Thanks for your help.

(In reply to comment #17)
> (In reply to comment #2)
> > Before this change is was always: 
> > 1. Go to host:/tmp/xxx.log
> > Now it's 
> > 1. Go to engine-host:/var/log/ovirt-engine/host-deploy and if it's not there
> > then,
> > 2. Go to host:/tmp/xxx.log
> > 
> > Is that correct?
> 
> Hey Alon,
> 
> I'm trying to resolve BZ#889333, and update "Red Hat Enterprise
> Virtualization Host Log Files"
> 
> https://access.redhat.com/knowledge/docs/en-US/
> Red_Hat_Enterprise_Virtualization/3.1/html-single/Installation_Guide/index.
> html#Red_Hat_Enterprise_Virtualization_Host_Log_Files1
> 
> Host side installation logs go to /tmp/xxx.log. What is the value of "xxx"?
> 
> Thanks for your help.

At host side it is on /tmp/ovirt-host-deploy-@DATE@.log, but once deployment is finished it is copied to engine side to /var/log/ovirt-engine/host-deploy/ovirt-@DATE@-@HOST@-@CORRELATION_ID@.log and removed from host.

Both logs' name is printed at event log during deployment.

Updated:

Red Hat Enterprise Virtualization Manager Log Files [7862, rev:374913]
Red Hat Enterprise Virtualization Host Log Files [9421, rev: 375129]

This bug is currently attached to errata RHEA-2013:14491. If this change is not to be documented in the text for this errata please either remove it from the errata, set the requires_doc_text flag to minus (-), or leave a "Doc Text" value of "--no tech note required" if you do not have permission to alter the flag.

Otherwise to aid in the development of relevant and accurate release documentation, please fill out the "Doc Text" field above with these four (4) pieces of information:

* Cause: What actions or circumstances cause this bug to present.

* Consequence: What happens when the bug presents.

* Fix: What was done to fix the bug.

* Result: What now happens when the actions or circumstances above occur. (NB: this is not the same as 'the bug doesn't present anymore')

Once filled out, please set the "Doc Type" field to the appropriate value for the type of change made and submit your edits to the bug.

For further details on the Cause, Consequence, Fix, Result format please refer to:

https://bugzilla.redhat.com/page.cgi?id=fields.html#cf_release_notes

Thanks in advance.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-0888.html