Created attachment 814434 [details]
/etc/ovirt-engine/notifier/log4j.xml

Created attachment 814435 [details]
/etc/ovirt-engine/engine-manage-domains/log4j.xml

Created attachment 814436 [details]
/etc/ovirt-engine/engine-config/log4j.xml

all attached files are stock, there were no modifications to them since setup time.

There are few things worthy to highlight:

1) the modification time of all of above files was RHEV-M installation time

2) the DEBUG messages started appearing in server.log 2 days after installation time

3) a snippet from /etc/ovirt-engine/notifier/log4j.xml:
      <!-- All the application server messages go here: -->
      <file-handler name="SERVER">
        <formatter>
          <pattern-formatter pattern="%d %-5p [%c] (%t) %s%E%n"/>
        </formatter>
        <file path="$getstring('ENGINE_LOG')/server.log"/>
        <append value="true"/>
      </file-handler>

      <!-- Only the engine messages go here: -->
      <file-handler name="ENGINE" autoflush="true">
        <level name="INFO"/>
        <formatter>
          <pattern-formatter pattern="%d %-5p [%c] (%t) %s%E%n"/>
        </formatter>
        <file path="$getstring('ENGINE_LOG')/engine.log"/>
        <append value="true"/>
      </file-handler>
--> engine.log has to be separately set up to log more verbosely than INFO, server.log does not. Why?

This is strange, I do not see any DEBUG message at my installations both rhel and fedora.

But I can try to force it to INFO, for logrotate, you are correct this server.log log should not have been appended, I will add logrotate.

the notifier, engine-config, engine-manage-domains configuration does relate to server.log behavior.

I do not see engine.log at /etc/ovirt-engine/notifier/log4j.xml, do you sure this is the case?

I was wrong, we do rotate the server.log

/etc/logrotate.d/ovirt-engine
---
"/var/log/ovirt-engine/engine.log" "/var/log/ovirt-engine/server.log" {
<snip>
---

> I was wrong, we do rotate the server.log

the log was rotated, I edited the subject but not the body. :/ BTW the rotation could use xz instead of gz and compress the most recent archived log as well.

> I do not see engine.log at /etc/ovirt-engine/notifier/log4j.xml, do you sure this is the case?

It was in /usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.xml.in , I managed to c&p wrong file name.

Using gz or xz for text is not that important. And we do not enforce compression, but let admin to specify whatever he likes at /etc/logrotate.conf

If debug started two days after installation it is probably not the configuration file setting that caused jboss to switch into debug.

Please try to restart and see if DEBUG appears, please try to reproduce.

(In reply to Alon Bar-Lev from comment #10)
> Using gz or xz for text is not that important. 

when the log gets big, it helps when one compression method saves two thirds of the other.

> And we do not enforce
> compression, but let admin to specify whatever he likes at
> /etc/logrotate.conf
> 
> If debug started two days after installation it is probably not the
> configuration file setting that caused jboss to switch into debug.
> 

I think so, as DEBUG was not specified in any relevant config file.

> Please try to restart and see if DEBUG appears, please try to reproduce.

It does unless I add <level name="INFO"/> statement to server.log file-handler entry.

(In reply to David Jaša from comment #11)
> (In reply to Alon Bar-Lev from comment #10)
> > Using gz or xz for text is not that important. 
> 
> when the log gets big, it helps when one compression method saves two thirds
> of the other.

Once again, this is not our call. logrotate is system component that is configured by system admin.

> 
> > And we do not enforce
> > compression, but let admin to specify whatever he likes at
> > /etc/logrotate.conf
> > 
> > If debug started two days after installation it is probably not the
> > configuration file setting that caused jboss to switch into debug.
> > 
> 
> I think so, as DEBUG was not specified in any relevant config file.
> 
> > Please try to restart and see if DEBUG appears, please try to reproduce.
> 
> It does unless I add <level name="INFO"/> statement to server.log
> file-handler entry.

So I guess something had changed in jboss, any upgrade related that?

(In reply to Alon Bar-Lev from comment #12)
> (In reply to David Jaša from comment #11)
> > (In reply to Alon Bar-Lev from comment #10)
> > > Using gz or xz for text is not that important. 
> > 
> > when the log gets big, it helps when one compression method saves two thirds
> > of the other.
> 
> Once again, this is not our call. logrotate is system component that is
> configured by system admin.
> 

logrotate is configured by packages as well and having good default configuration is their responsibility as well

> So I guess something had changed in jboss, any upgrade related that?

going through the logs, there was jboss update first, then engine clean up, then engine fresh installation. The debug priority in logs appeared two minutes after jboss update finished.

(In reply to David Jaša from comment #13)
> (In reply to Alon Bar-Lev from comment #12)
> > (In reply to David Jaša from comment #11)
> > > (In reply to Alon Bar-Lev from comment #10)
> > > > Using gz or xz for text is not that important. 
> > > 
> > > when the log gets big, it helps when one compression method saves two thirds
> > > of the other.
> > 
> > Once again, this is not our call. logrotate is system component that is
> > configured by system admin.
> > 
> 
> logrotate is configured by packages as well and having good default
> configuration is their responsibility as well

If admin want to overwrite our configuration he may do so, as he can do for global configuration. We should introduce as minimum customization (and dependencies) as possible.

> > So I guess something had changed in jboss, any upgrade related that?
> 
> going through the logs, there was jboss update first, then engine clean up,
> then engine fresh installation. The debug priority in logs appeared two
> minutes after jboss update finished.

So jboss broke their own log level inheritance model... second time they break log related, previous was the removal of the option to specify setting via java properties in log4j.

*** Bug 1021561 has been marked as a duplicate of this bug. ***

ok, is20.2

Closing - RHEV 3.3 Released

Closing - RHEV 3.3 Released

Closing - RHEV 3.3 Released