Bug 1866123

Summary:	Ironic-inspector logs are not saved to ramdisk
Product:	OpenShift Container Platform	Reporter:	Andrew Kiselev <akiselev>
Component:	Bare Metal Hardware Provisioning	Assignee:	Bob Fournier <bfournie>
Bare Metal Hardware Provisioning sub component:	ironic	QA Contact:	Lubov <lshilin>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	medium
Priority:	medium	CC:	afasano, akiselev, bfournie, dtantsur, tsedovic
Version:	4.6	Keywords:	Triaged, UpcomingSprint
Target Milestone:	---
Target Release:	4.6.0
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	The ironic image was using an incorrect name in the ironic inspector 'extra_kernel_params'. This was resulting in the inspector logs not being generated. The fix was to use the correct name, and then the logs are properly generated.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-10-27 16:24:53 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Andrew Kiselev 2020-08-04 22:46:30 UTC

Description of problem:
Ironic-inspector logs are not saved to the ramdisk even when configured to do so. I don't see a copy of these logs at /shared/log/ironic-inspector/ramdisk in ironic-inspector container.


Version-Release number of selected component (if applicable):
This problem exists in the latest 4.6 and 4.5.4 

How reproducible:
100% reproducible. 

Steps to Reproduce:
1.Deploy target OCP image (dev-scripts or baremetal). Keep bootstrap VM (export OPENSHIFT_INSTALL_PRESERVE_BOOTSTRAP=y)
2. Log in to the the container from the bootstrap (sudo podman exec -it ironic-inspector bash). the logs directory structure was a little different and empty:
3. Observe /shared/log/ironic-inspector/ramdisk is missing.
4. Observe ramdisk warning log messages from the bootstrap: sudo podman logs ironic-inspector | grep ramdisk

2020-07-27 22:03:01.470 1 DEBUG ironic_inspector.process [-] [node: 2735e0d6-d981-4ec4-949e-0d9d9e63c321 state processing MAC 00:89:2f:4f:41:2f] Running post-processing hook ramdisk_error _run_post_hooks /usr/lib/python3.6/site-packages/ironic_inspector/process.py:268
2020-07-27 22:03:01.886 1 WARNING ironic_inspector.plugins.extra_hardware [-] [node: 2735e0d6-d981-4ec4-949e-0d9d9e63c321 state processing MAC 00:89:2f:4f:41:2f] No extra hardware information was received from the ramdisk
2020-07-27 22:03:01.899 1 WARNING ironic_inspector.process [-] [node: 2735e0d6-d981-4ec4-949e-0d9d9e63c321 MAC 00:89:2f:4f:41:2f] No logs were passed by the ramdisk

Actual results:
/shared/log/ironic-inspector/ramdisk is missing.

Expected results:
/shared/log/ironic-inspector/ramdisk should store collected logs.

Additional info:
https://coreos.slack.com/archives/CFP6ST0A3/p1595955724077000

Comment 2 Bob Fournier 2020-08-17 11:54:24 UTC

Fix has merged.

Comment 5 Bob Fournier 2020-08-18 14:47:37 UTC

Changing Status until patch merges downstream to https://github.com/openshift/ironic-image.

Comment 7 Lubov 2020-08-23 08:14:09 UTC

[core@localhost ~]$ sudo podman exec -it ironic-inspector bash
[root@localhost /]# ls /shared/log/ironic-inspector/ramdisk/
0f731aab-f8a8-412c-be67-9f7d6d5ac2f7_20200823-080041.777466.tar.gz  d5349540-02b2-451f-b3bf-d81068b3013a_20200823-080040.818676.tar.gz
37a43eba-bbe9-4365-86d8-fbba4c5fc79e_20200823-080041.300119.tar.gz 

Verified on
Client Version: 4.6.0-0.nightly-2020-08-22-221008
Server Version: 4.6.0-0.nightly-2020-08-22-221008
Kubernetes Version: v1.19.0-rc.2+3e083ac-dirty

Comment 9 errata-xmlrpc 2020-10-27 16:24:53 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196