Bug 875949 - teardown_pam_fs_limits.sh may fail on nodes with NFS mounts
Summary: teardown_pam_fs_limits.sh may fail on nodes with NFS mounts
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OKD
Classification: Red Hat
Component: Containers
Version: 2.x
Hardware: Unspecified
OS: Unspecified
medium
high
Target Milestone: ---
: ---
Assignee: Jhon Honce
QA Contact: libra bugs
URL:
Whiteboard:
Depends On: 872379
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-11-12 22:21 UTC by Jhon Honce
Modified: 2015-05-14 23:02 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of: 872379
Environment:
Last Closed: 2012-12-19 19:27:42 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Jhon Honce 2012-11-12 22:21:28 UTC
+++ This bug was initially created as a clone of Bug #872379 +++

Description of problem:
If a node has any NFS mounts, the script may loop infinitely looking for the correct dev for GEAR_BASE_DIR.

Version-Release number of selected component (if applicable):
2.0.19

How reproducible:
Depends on order of mounts provided by df command

Steps to Reproduce:
1. Add NFS mount to a node
2. Create a gear
3.
  
Actual results:
If NFS mount is reported after correct mount script will loop infinitely

Expected results:
Script should ignore all NFS mounts as GEAR_BASE_DIR on an NFS mount is not supported.

Additional info:

--- Additional comment from jhonce on 2012-11-01 17:01:01 MST ---

Created attachment 636871 [details]
Screen shot providing additional information

--- Additional comment from jhonce on 2012-11-01 17:01:26 MST ---

Created attachment 636872 [details]
Screen shot providing additional information

--- Additional comment from jhonce on 2012-11-01 17:01:48 MST ---

Created attachment 636873 [details]
Screen shot providing additional information

--- Additional comment from rmillner on 2012-11-09 17:18:24 MST ---

The part that was failing turned out to be dead code anyway (results unused).  Cleaned up the script.

Pull Request:
https://github.com/openshift/origin-server/pull/866

--- Additional comment from jhou on 2012-11-12 04:51:32 MST ---

To verify this bug, QE need to mount an NFS on node, which may not be easy on a devenv instance.

So we verified this bug on our on-premise environment using above fix

Steps:
1. mount an NFS to node
2. rum setup_pam_fs_limits.sh 

[root@node openshift]# /bin/sh -x /usr/libexec/openshift/lib/setup_pam_fs_limits.sh 5abd74eee31d4008bf78185c486b9e64 0 0 0
+ LIMITSVARS='core data fsize memlock nofile rss stack cpu nproc as maxlogins priority locks sigpending msgqueue nice rprio'
+ source /etc/openshift/node.conf
++ PUBLIC_HOSTNAME=node.example.com
++ PUBLIC_IP=10.66.79.5
++ BROKER_HOST=broker.example.com
++ CLOUD_DOMAIN=example.com
++ GEAR_BASE_DIR=/var/lib/openshift
++ GEAR_SKEL_DIR=/etc/openshift/skel
++ GEAR_SHELL=/usr/bin/oo-trap-user
++ GEAR_GECOS='OpenShift guest'
++ GEAR_MIN_UID=500
++ GEAR_MAX_UID=6500
++ OPENSHIFT_NODE_PLUGINS=openshift-origin-node/plugins/unix_user_observer
++ CARTRIDGE_BASE_PATH=/usr/libexec/openshift/cartridges
++ LAST_ACCESS_DIR=/var/lib/openshift/.last_access
++ APACHE_ACCESS_LOG=/var/log/httpd/access_log
++ PROXY_MIN_PORT_NUM=35531
++ PROXY_PORTS_PER_GEAR=5
++ CREATE_APP_SYMLINKS=1
++ OPENSHIFT_HTTP_CONF_DIR=/etc/httpd/conf.d/openshift
+ limits_order=84
+ limits_nproc=100
+ quota_files=1000
++ expr 1024 '*' 128
+ quota_blocks=131072
+ source /etc/openshift/resource_limits.conf
++ node_profile=small
++ quota_files=40000
++ quota_blocks=1048576
++ max_apps=100
++ max_active_apps=100
++ limits_nproc=250
++ cpu_shares=128
++ cpu_cfs_quota_us=30000
++ memory_limit_in_bytes=536870912
++ memory_memsw_limit_in_bytes=641728512
++ apache_bandwidth='all 500000'
++ apache_maxconnection='all 20'
++ apache_bandwidtherror=510
++ rotatelogs_interval=86400
++ rotatelogs_format=-%Y%m%d-%H%M%S-%Z
+ username=5abd74eee31d4008bf78185c486b9e64
+ quota_blocks_custom=0
+ quota_files_custom=0
+ nproc_custom=0
+ '[' -n 0 ']'
+ '[' 0 -gt 1048576 ']'
+ '[' -n 0 ']'
+ '[' 0 -gt 40000 ']'
+ '[' -n 0 ']'
+ '[' 0 -le 250 ']'
+ limits_nproc=0
+ set_pam_limits 5abd74eee31d4008bf78185c486b9e64
+ USERNAME=5abd74eee31d4008bf78185c486b9e64
+ LIMITSFILE=/etc/security/limits.d/84-5abd74eee31d4008bf78185c486b9e64.conf
+ '[' -z '' ']'
+ cat
+ for KEY in '$LIMITSVARS'
++ eval echo '$limits_core'
+++ echo
+ VALUE=
+ '[' -n '' ']'
+ for KEY in '$LIMITSVARS'
++ eval echo '$limits_data'
+++ echo
+ VALUE=
+ '[' -n '' ']'
+ for KEY in '$LIMITSVARS'
++ eval echo '$limits_fsize'
+++ echo
+ VALUE=
+ '[' -n '' ']'
+ for KEY in '$LIMITSVARS'
++ eval echo '$limits_memlock'
+++ echo
+ VALUE=
+ '[' -n '' ']'
+ for KEY in '$LIMITSVARS'
++ eval echo '$limits_nofile'
+++ echo
+ VALUE=
+ '[' -n '' ']'
+ for KEY in '$LIMITSVARS'
++ eval echo '$limits_rss'
+++ echo
+ VALUE=
+ '[' -n '' ']'
+ for KEY in '$LIMITSVARS'
++ eval echo '$limits_stack'
+++ echo
+ VALUE=
+ '[' -n '' ']'
+ for KEY in '$LIMITSVARS'
++ eval echo '$limits_cpu'
+++ echo
+ VALUE=
+ '[' -n '' ']'
+ for KEY in '$LIMITSVARS'
++ eval echo '$limits_nproc'
+++ echo 0
+ VALUE=0
+ '[' -n 0 ']'
+ '[' -z '' ']'
+ echo '5abd74eee31d4008bf78185c486b9e64 hard nproc 0'
+ for KEY in '$LIMITSVARS'
++ eval echo '$limits_as'
+++ echo
+ VALUE=
+ '[' -n '' ']'
+ for KEY in '$LIMITSVARS'
++ eval echo '$limits_maxlogins'
+++ echo
+ VALUE=
+ '[' -n '' ']'
+ for KEY in '$LIMITSVARS'
++ eval echo '$limits_priority'
+++ echo
+ VALUE=
+ '[' -n '' ']'
+ for KEY in '$LIMITSVARS'
++ eval echo '$limits_locks'
+++ echo
+ VALUE=
+ '[' -n '' ']'
+ for KEY in '$LIMITSVARS'
++ eval echo '$limits_sigpending'
+++ echo
+ VALUE=
+ '[' -n '' ']'
+ for KEY in '$LIMITSVARS'
++ eval echo '$limits_msgqueue'
+++ echo
+ VALUE=
+ '[' -n '' ']'
+ for KEY in '$LIMITSVARS'
++ eval echo '$limits_nice'
+++ echo
+ VALUE=
+ '[' -n '' ']'
+ for KEY in '$LIMITSVARS'
++ eval echo '$limits_rprio'
+++ echo
+ VALUE=
+ '[' -n '' ']'
+ set_fs_quotas 5abd74eee31d4008bf78185c486b9e64 1048576 40000
+ quotas_enabled /var/lib/openshift
++ get_mountpoint /var/lib/openshift
++ awk '{ print $6 }'
++ tail -1
++ df -P /var/lib/openshift
+ QUOTA_ROOT=/
+ '[' -z / ']'
+ quotaon -u -p /
+ '[' 1 -eq 0 ']'
+ return 0
++ get_mountpoint /var/lib/openshift
++ awk '{ print $6 }'
++ tail -1
++ df -P /var/lib/openshift
+ setquota 5abd74eee31d4008bf78185c486b9e64 0 1048576 0 40000 /
[root@node openshift]# echo $?
0
 
There is not errors during execution, so move this bug to verified.

Comment 1 Jhon Honce 2012-11-12 23:27:40 UTC
Reproduce:

1) yum install nfs
2) mkdir /share
3) add /share  *(rw,no_root_squash) to /etc/exports
3) service rpcbind start
4) service nfs start
5) exportfs -a
6) mount -t nfs localhost:/share /mnt

7) Create gear
8) Delete gear


Fixed in https://github.com/openshift/origin-server/pull/877

Comment 2 Jianwei Hou 2012-11-13 05:55:20 UTC
Verified on devenv_2458

1) yum install nfs-utils
2) mkdir /share
3) add /share  *(rw,no_root_squash) to /etc/exports
3) service rpcbind start
4) service nfs start
5) exportfs -a
6) mount -t nfs localhost:/share /mnt
7) Create gear
rhc app create php1 php-5.3, gear is created successfully
8) Delete gear
rhc app delete php1, gear is deleted successfully


Note You need to log in before you can comment on or make changes to this bug.