Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1309764

Summary: ipsec barf does not show pluto log correctly in the output
Product: Red Hat Enterprise Linux 7 Reporter: Jaroslav Aster <jaster>
Component: libreswanAssignee: Paul Wouters <pwouters>
Status: CLOSED ERRATA QA Contact: Jaroslav Aster <jaster>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.3CC: jaster, lmiksik, omoris, pwouters
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Cause: ipsec barf missed support for journald Consequence: ipsec barf tries to find system logs to include in a report it generates. these reports would lack the log entries normally includes Fix: detect when journald is used and use journalctl to get the log entries for ipsec Result: ipsec barf shows log entries in its output
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-03 21:22:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jaroslav Aster 2016-02-18 16:01:25 UTC
Description of problem:

ipsec barf does not show pluto log correctly in the output. It shows all lines related to all pluto's runs from pluto log, because there is bad grep pattern 'Starting Pluto subsystem' in /usr/libexec/ipsec/barf. This pattern is not part of pluto log file on rhel-7 anymore. ipsec barf on rhel-6 works fine.

Version-Release number of selected component (if applicable):

libreswan-3.15-5.el7_1

How reproducible:

100%

Steps to Reproduce:

rhel-7:

# grep -n 'Starting Pluto subsystem' /var/log/secure

# grep -n 'Starting Pluto' /var/log/secure
59:Feb 18 09:31:24 qeos-37 pluto[14452]: Starting Pluto (Libreswan Version 3.15 XFRM(netkey) KLIPS NSS DNSSEC FIPS_CHECK LABELED_IPSEC LIBCAP_NG LINUX_AUDIT XAUTH_PAM NETWORKMANAGER CURL(non-NSS) LDAP(non-NSS)) pid:14452
2560:Feb 18 10:07:29 qeos-37 pluto[15707]: Starting Pluto (Libreswan Version 3.15 XFRM(netkey) KLIPS NSS DNSSEC FIPS_CHECK LABELED_IPSEC LIBCAP_NG LINUX_AUDIT XAUTH_PAM NETWORKMANAGER CURL(non-NSS) LDAP(non-NSS)) pid:15707

You can see that first, old, 'Starting Pluto' is on line 59 and the second, current, is on line 2560.

# ipsec barf show
...
+ _________________________ plog
+ egrep -i pluto
+ case "$1" in
+ cat
+ sed -n '59,$p' /var/log/secure
Feb 18 09:31:24 qeos-37 pluto[14452]: Starting Pluto (Libreswan Version 3.15 XFRM(netkey) KLIPS NSS DNSSEC FIPS_CHECK LABELED_IPSEC LIBCAP_NG LINUX_AUDIT XAUTH_PAM NETWORKMANAGER CURL(non-NSS) LDAP(non-NSS)) pid:14452
...

Actual results:

ipsec barf shows all lines related to all pluto's runs from pluto log.

Expected results:

ipsec barf shows lines only for current run from pluto log.

Additional info:

Comment 1 Paul Wouters 2016-02-19 14:18:04 UTC
This is due to _plutorun not being used with systemd (which printed the "subsystem" line). the pluto daemon itself prints "Starting Pluto", so the barf command should grep for that instead to match either case.

A fix was pushed upstream. It would come into 7.3 via a rebase.

Comment 3 Jaroslav Aster 2016-07-20 13:48:15 UTC
Hi Paul,

unfortunately, this bug is not fixed. It does not show last ipsec's log only. Switching it to assigned state. 


You added journalctl command to show last ipsec's log in ipsec barf command.

journalctl -u ipsec.service --since '1 hour ago'

This command shows all ipsec's logs newer then one hour, but you want to show only last ipsec's log. If ipsec runs longer than one hour, journalctl does not show all log. If ipsec runs less then one hour and there was at least one previous ipsec's run, journalctl shows more then last log.

There is another issue with journalctl. journalctl has a default pager, so if the log is longer then one page, default pager is run and you can not see what ipsec barf printed before. You have to use --no-pager option to avoid journalctl's pager.

You can use this command to show last ipsec's log. I know it is not elegant, but it illustrates what I mean.

last_ipsec_run="$(date -d "$(grep 'Starting Internet Key Exchange (IKE) Protocol Daemon for IPsec...' /var/log/messages | tail -n 1 | cut -d ' ' -f 1,2,3)" '+%Y-%m-%d %H:%M:%S')"
journalctl --unit ipsec.service --no-page --since "$last_ipsec_run"

I see, there is a fallback on systems where journalctl does not exist or it is not executable. It uses sed. This fallback does not seem be fixed too. It shows much more information than journalctl version.

Comment 4 Paul Wouters 2016-07-22 13:18:46 UTC
There can be arbitrary many logs in the system for ipsec. We cannot just go back and list all the logs. the "barf" command is meant to run as a test tool for enduers to give the developers a snapshot of the system and what happened.

If you want all the logs, you should configure logfile=/some/pluto.log instead and barf won't do the log part, or you can run journalctl manually and decide how far you want to look back.

if systemd adds an option to journalctl to show "logs since start of service" then we can change this behaviour to use that. but the man page of journalctl did not show me that option.

Comment 5 Jaroslav Aster 2016-08-04 13:21:15 UTC
(In reply to Paul Wouters from comment #4)
> There can be arbitrary many logs in the system for ipsec. We cannot just go
> back and list all the logs. the "barf" command is meant to run as a test
> tool for enduers to give the developers a snapshot of the system and what
> happened.

Understand, but I reported this ticket, because ipsec barf on rhel-7 does not show logs for last ipsec run as it shows on rhel-6. In current release, it still does not show logs for the last ipsec run, so for me, this bug is not fixed. If you think it is not a bug, we can close the ticket, without that, I can not verify this bug, because reported issue is not fixed.
 
> If you want all the logs, you should configure logfile=/some/pluto.log
> instead and barf won't do the log part, or you can run journalctl manually
> and decide how far you want to look back.

I want ipsec barf to work as it works on rhel-6.

> if systemd adds an option to journalctl to show "logs since start of
> service" then we can change this behaviour to use that. but the man page of
> journalctl did not show me that option.

No, journalctl does not have this option, but I gave you a workaround how to find out the time when last ipsec has started.

last_ipsec_run="$(date -d "$(grep 'Starting Internet Key Exchange (IKE) Protocol Daemon for IPsec...' /var/log/messages | tail -n 1 | cut -d ' ' -f 1,2,3)" '+%Y-%m-%d %H:%M:%S')"

Then you can use this datetime in journalctl.

journalctl --unit ipsec.service --no-page --since "$last_ipsec_run"

Whether or not you will decide use this, you have to add, at least, --no-page option, because without that journalctl starts pager and pager removes all previous ipsec barf output. It's stupid, but it's sytstemd :-(. You can test it yourself with ipsec logs longer than one page.

Comment 6 Paul Wouters 2016-08-04 15:41:02 UTC
Why are you assuming /var/log/messages exists on systemd systems? I think that is only true if someone installed rsyslog along with systemd.

ipsec barf shows for me near the end:

+ dmesg
+ egrep -i 'klips|ipsec'
[89888.010280] IPv4 over IPsec tunneling driver
+ _________________________ plog
+ test -x /usr/bin/journalctl
+ journalctl -u ipsec.service --since '1 hour ago'
-- Logs begin at Mon 2016-08-01 15:44:00 EDT, end at Thu 2016-08-04 11:31:06 EDT. --
+ _________________________ date
+ date
Thu Aug  4 11:32:51 EDT 2016

which seems to match journalctl -u ipsec.service --since '1 hour ago'

# journalctl --unit ipsec.service  --since '1 hour ago'
-- Logs begin at Mon 2016-08-01 15:44:00 EDT, end at Thu 2016-08-04 11:31:06 EDT. --

It shows no logs because I have logfile= set. When I unset that, syslog is used. It shows a lot of data and "ipsec barf" ends on the journalctl pager. But redirecting with "ipsec barf > out" works properly and all logs show up in the file including all the parts before the journalctl command is issued:

head -10 out

root@bofh7:/var/www/cgi-bin# head -10 out
bofh7.nohats.ca
Thu Aug  4 11:35:32 EDT 2016
+ _________________________ version
+ ipsec --version
Linux Libreswan 3.15 (netkey) on 3.10.0-327.22.2.el7.x86_64
+ _________________________ /proc/version
+ cat /proc/version
Linux version 3.10.0-327.22.2.el7.x86_64 (builder.centos.org) (gcc version 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC) ) #1 SMP Thu Jun 23 17:05:11 UTC 2016
+ _________________________ /proc/net/ipsec_eroute
+ test -r /proc/net/ipsec_eroute


So I cannot reproduce your problem, and I think we cannot add any code that relies on /var/log/messages being present. I think a bug/feature request can be send to systemd to ask for "all logs since last service start", however that would also not be ideal to use, for instance if the pluto daemon crashes, and you want to send us a bunch of debug logs, you would want it to cover before the crash as well, so I believe saying "the last hour" is better.

Comment 7 Jaroslav Aster 2016-08-11 11:19:19 UTC
(In reply to Paul Wouters from comment #6)
> Why are you assuming /var/log/messages exists on systemd systems? I think
> that is only true if someone installed rsyslog along with systemd.

Good point. You are right.

> It shows a lot of data and "ipsec barf" ends on the journalctl pager.
> But redirecting with "ipsec barf > out" works properly and all logs show up
> in the file including all the parts before the journalctl command is issued:

Run pager inside program is stupid and it's against unix philosophy. It's up to users whether or not they use | and less/more. I know, I can redirect all output into the file or use '| less' and journalctl detects it, but I do not see any reason why restrict users to print _all_ ipsec barf output on the console if they want.

> So I cannot reproduce your problem, and I think we cannot add any code that
> relies on /var/log/messages being present. I think a bug/feature request can
> be send to systemd to ask for "all logs since last service start", however
> that would also not be ideal to use, for instance if the pluto daemon
> crashes, and you want to send us a bunch of debug logs, you would want it to
> cover before the crash as well, so I believe saying "the last hour" is
> better.

Ok. I agree with change ipsec barf's output. What you wrote makes sense, but I have to insist on '--no-pager' option.

Comment 8 Jaroslav Aster 2016-08-11 13:55:44 UTC
Two notes:

1,

It would be fine to have updated ipsec_barf man-page. There is a description how ipsec barf works, but there is no mention about journalctl.

2,

It is not redhat-7 issue, but there is a fall-back for systems which do not use systemd. This fall-back has not been changed in the new version, so it still uses string 'Starting Pluto subsystem' for grep, see first comment. Is it ok?

Comment 13 errata-xmlrpc 2016-11-03 21:22:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2603.html