Bug 1574866 - [RFE] Monitor of /var/log with external mount for DiskPressure on the node
Summary: [RFE] Monitor of /var/log with external mount for DiskPressure on the node
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: RFE
Version: 3.7.0
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: ---
Assignee: Derek Carr
QA Contact: Xiaoli Tian
: 1664234 1730042 (view as bug list)
Depends On:
TreeView+ depends on / blocked
Reported: 2018-05-04 08:18 UTC by Sudarshan Chaudhari
Modified: 2021-06-10 16:04 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2019-06-12 11:55:26 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 3428961 0 Performance tune None Error syncing pod on Openshift node 2019-07-22 20:15:16 UTC

Internal Links: 1730042

Comment 1 Sudarshan Chaudhari 2018-05-04 09:15:00 UTC
Description of problem:

For the DiskPressure criteria, the /var/log need to be monitored if the /var/logs is mounted with some mount point. 

If the /var/log is mounted at say: /dev/sdb2 and /var is mounted at say: /dev/sdb1

[root@master]# df -h
Filesystem                    Size  Used Avail Use% Mounted on
/dev/mapper/rhel_root         29G   2.8G   26G  10% /
/dev/sdb1                     10G  1020M  8.9G  10% /var
/dev/sdb2                     6G      6G  126K 100% /var/log

- "oc describe node master" shows:
[root@master byo]# oc describe node master
Name:			master
. . .
  Type			Status	LastHeartbeatTime			LastTransitionTime			Reason				Message
  ----			------	-----------------			------------------			------				-------
  OutOfDisk 		False 	Fri, 04 May 2018 12:50:41 +0530 	Fri, 04 May 2018 11:35:33 +0530 	KubeletHasSufficientDisk 	kubelet has sufficient disk space available
  MemoryPressure 	False 	Fri, 04 May 2018 12:50:41 +0530 	Fri, 04 May 2018 11:35:33 +0530 	KubeletHasSufficientMemory 	kubelet has sufficient memory available
  DiskPressure 		False 	Fri, 04 May 2018 12:50:41 +0530 	Fri, 04 May 2018 11:35:33 +0530 	KubeletHasNoDiskPressure 	kubelet has no disk pressure
  Ready 		True 	Fri, 04 May 2018 12:50:41 +0530 	Fri, 04 May 2018 11:43:15 +0530 	KubeletReady 			kubelet is posting ready status

- "oc describe pod" shows
  FirstSeen	LastSeen	Count	From					SubObjectPath	Type		Reason		Message
  ---------	--------	-----	----					-------------	--------	------		-------
  1m		1m		1	default-scheduler					Normal		Scheduled	Successfully assigned registry-console-10-deploy to node1.example.com
  1m		5s		9	kubelet, node1.example.com			Warning		FailedSync	Error syncing pod

- "journalctl" on the node shows:
MESSAGE=E0321 11:59:06.703825   91791 pod_workers.go:182] Error syncing pod fdd4f587-2d13-11e8-9f0e-00505682f47f ("registry-console-10-deploy(fdd4f587-2d13-11e8-9f0e-00505682f47f)"), skipping: failed to "CreatePodSandbox" for "registry-console-10-deploy(fdd4f587-2d13-11e8-9f0e-00505682f47f)" with CreatePodSandboxError: "Create pod log directory for pod \"registry-console-10-deploy(fdd4f587-2d13-11e8-9f0e-00505682f47f)\" failed: mkdir /var/log/pods/fdd4f587-2d13-11e8-9f0e-00505682f47f: no space left on device"

How reproducible:


Actual results:

The DiskPressure Conditions should also check the /var/log if mounted with external drive

Expected results:

The Kubelet should report Node under disk pressure.

Comment 7 Kirsten Newcomer 2019-06-12 11:55:26 UTC
With the introduction of OpenShift 4, Red Hat has delivered or roadmapped a substantial number of features based on feedback by our customers.  Many of the enhancements encompass specific RFEs which have been requested, or deliver a comparable solution to a customer problem, rendering an RFE redundant.

This bz (RFE) has been identified as a feature request not yet planned or scheduled for an OpenShift release and is being closed. 

If this feature is still an active request that needs to be tracked, Red Hat Support can assist in filing a request in the new JIRA RFE system, as well as provide you with updates as the RFE progress within our planning processes. Please open a new support case: https://access.redhat.com/support/cases/#/case/new 

Opening a New Support Case: https://access.redhat.com/support/cases/#/case/new 

As the new Jira RFE system is not yet public, Red Hat Support can help answer your questions about your RFEs via the same support case system.

Comment 8 Seth Jennings 2019-07-22 20:15:16 UTC
*** Bug 1664234 has been marked as a duplicate of this bug. ***

Comment 9 Seth Jennings 2019-07-23 16:31:19 UTC
*** Bug 1730042 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.