Bug 1146750 - cannot do anything in /tmp/ after running `rhc app tidy`
Summary: cannot do anything in /tmp/ after running `rhc app tidy`
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Online
Classification: Red Hat
Component: Containers
Version: 2.x
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 2.x
Assignee: Jhon Honce
QA Contact: libra bugs
URL:
Whiteboard:
Depends On:
Blocks: 1154763
TreeView+ depends on / blocked
 
Reported: 2014-09-26 01:53 UTC by Fraser Tweedale
Modified: 2015-05-14 23:37 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1154763 (view as bug list)
Environment:
Last Closed: 2015-02-18 16:51:21 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Fraser Tweedale 2014-09-26 01:53:24 UTC
Description of problem:

After running `rhc app tidy`, the ability to create files or directories
in /tmp/ is lost.  A "No such file or directory" error occurs, even though
/tmp/ does exist.


Version-Release number of selected component (if applicable):


How reproducible:

Occurs with diy cartridge and the Haskell community cartridges, which were
the only cartridges tested.


Steps to Reproduce:
1. Create an app.
2. ssh in and confirm that files and directories can be created under /tmp/
3. Run `rhc app tidy` for the app.
4. ssh in and confirm that files and directories cannot be created.

Actual results:

[diy-ftweedal.rhcloud.com 5424c3d14382ec6c8e0000b3]\> touch /tmp/foo
touch: cannot touch `/tmp/foo': No such file or directory
[diy-ftweedal.rhcloud.com 5424c3d14382ec6c8e0000b3]\> ls /tmp/
[diy-ftweedal.rhcloud.com 5424c3d14382ec6c8e0000b3]\> ls -ld /tmp/
drwxrwxrwt. 0 5424c3d14382ec6c8e0000b3 root 6 Sep 25 21:40 /tmp/


Expected results:

Files and directories can still be created under /tmp/ after running
`rhc app tidy`.


Additional info:

Comment 1 Lili Nader 2014-09-28 03:51:15 UTC
Works for me.  This is what I tried

1. create app
rhc app create -a diy -t diy
2. tidy app
rhc app tidy -a diy
3. ssh into gear and created test directory and file
mkdir /tmp/test
vi /tmp/test/test.txt
4. exit and run app tidy again
5. The file I created was deleted but I could create another file and directory.

Comment 2 Qixuan Wang 2014-09-28 05:23:50 UTC
Test on devenv_5196, diy and python cartridge, it works for me, too.

1. Create an app
# rhc app create py33
2. Create files and directories on the gear
# rhc ssh py33
[py33-domain.dev.rhcloud.com 5427d00dc4219bd2a9000031]\> mkdir /tmp/test
[py33-domain.dev.rhcloud.com 5427d00dc4219bd2a9000031]\> touch /tmp/test.txt
[py33-domain.dev.rhcloud.com 5427d00dc4219bd2a9000031]\> ls -la /tmp
total 12
drwxrwxrwt.  3 5427d00dc4219bd2a9000031 root                     4096 Sep 28 05:10 .
dr-xr-xr-x. 26 root                     root                     4096 Sep 28 03:25 ..
drwx------.  2 5427d00dc4219bd2a9000031 5427d00dc4219bd2a9000031 4096 Sep 28 05:09 test
-rw-------.  1 5427d00dc4219bd2a9000031 5427d00dc4219bd2a9000031    0 Sep 28 05:10 test.txt
srwx------.  1 5427d00dc4219bd2a9000031 5427d00dc4219bd2a9000031    0 Sep 28 05:08 wsgi.25127.0.1.sock
3. Tidy the app
# rhc app tidy py33
4. Check the files and directories on the gear, they were deleted
[py33-domain.dev.rhcloud.com 5427d00dc4219bd2a9000031]\> ls -la /tmp
total 8
drwxrwxrwt.  2 5427d00dc4219bd2a9000031 root                     4096 Sep 28 05:11 .
dr-xr-xr-x. 26 root                     root                     4096 Sep 28 03:25 ..
srwx------.  1 5427d00dc4219bd2a9000031 5427d00dc4219bd2a9000031    0 Sep 28 05:11 wsgi.27283.0.1.sock
5. Create files and directories on the gear again
[py33-domain.dev.rhcloud.com 5427d00dc4219bd2a9000031]\> touch /tmp/file
[py33-domain.dev.rhcloud.com 5427d00dc4219bd2a9000031]\> mkdir /tmp/dir
[py33-domain.dev.rhcloud.com 5427d00dc4219bd2a9000031]\> ls -la /tmp
total 12
drwxrwxrwt.  3 5427d00dc4219bd2a9000031 root                     4096 Sep 28 05:12 .
dr-xr-xr-x. 26 root                     root                     4096 Sep 28 03:25 ..
drwx------.  2 5427d00dc4219bd2a9000031 5427d00dc4219bd2a9000031 4096 Sep 28 05:12 dir
-rw-------.  1 5427d00dc4219bd2a9000031 5427d00dc4219bd2a9000031    0 Sep 28 05:12 file
srwx------.  1 5427d00dc4219bd2a9000031 5427d00dc4219bd2a9000031    0 Sep 28 05:11 wsgi.27283.0.1.sock

The result is what we expect.

Comment 3 Fraser Tweedale 2014-09-30 06:22:28 UTC
This is definitely still occurring for me.  Transcript with python-3.3 cartridge::


[dhcp-40-8:~/doc/blog] (ruby-2.1.2) [ master ] ftweedal% rhc app create py33 python-3.3 --no-git
Application Options
-------------------
Domain:     ftweedal
Cartridges: python-3.3
Gear Size:  default
Scaling:    no

Creating application 'py33' ... done


Waiting for your DNS name to be available ... done

Your application 'py33' is now available.

  URL:        http://py33-ftweedal.rhcloud.com/
  SSH to:     542a4b0f500446f46e000480.com
  Git remote: ssh://542a4b0f500446f46e000480.com/~/git/py33.git/

Run 'rhc show-app py33' for more details about your app.
[dhcp-40-8:~/doc/blog] (ruby-2.1.2) [ master ] ftweedal% rhc ssh py33
Connecting to 542a4b0f500446f46e000480.com ...
The authenticity of host 'py33-ftweedal.rhcloud.com (54.221.166.63)' can't be established.
RSA key fingerprint is cf:ee:77:cb:0e:fc:02:d7:72:7e:ae:80:c0:90:88:a7.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'py33-ftweedal.rhcloud.com,54.221.166.63' (RSA) to the list of known hosts.

    *********************************************************************

    You are accessing a service that is for use only by authorized users.
    If you do not have authorization, discontinue use at once.
    Any use of the services is subject to the applicable terms of the
    agreement which can be found at:
    https://www.openshift.com/legal

    *********************************************************************

    Welcome to OpenShift shell

    This shell will assist you in managing OpenShift applications.

    !!! IMPORTANT !!! IMPORTANT !!! IMPORTANT !!!
    Shell access is quite powerful and it is possible for you to
    accidentally damage your application.  Proceed with care!
    If worse comes to worst, destroy your application with "rhc app delete"
    and recreate it
    !!! IMPORTANT !!! IMPORTANT !!! IMPORTANT !!!

    Type "help" for more info.


[py33-ftweedal.rhcloud.com 542a4b0f500446f46e000480]\> mkdir /tmp/test
[py33-ftweedal.rhcloud.com 542a4b0f500446f46e000480]\> touch /tmp/test.txt
[py33-ftweedal.rhcloud.com 542a4b0f500446f46e000480]\> ls -la /tmp
total 12
drwxrwxrwt.  3 542a4b0f500446f46e000480 root                     4096 Sep 30 02:19 .
dr-xr-xr-x. 26 root                     root                     4096 Sep 25 04:42 ..
drwx------.  2 542a4b0f500446f46e000480 542a4b0f500446f46e000480 4096 Sep 30 02:19 test
-rw-------.  1 542a4b0f500446f46e000480 542a4b0f500446f46e000480    0 Sep 30 02:19 test.txt
srwx------.  1 542a4b0f500446f46e000480 542a4b0f500446f46e000480    0 Sep 30 02:18 wsgi.493463.0.1.sock
[py33-ftweedal.rhcloud.com 542a4b0f500446f46e000480]\> exit
Shared connection to py33-ftweedal.rhcloud.com closed.
[dhcp-40-8:~/doc/blog] (ruby-2.1.2) [ master ] ftweedal% rhc app tidy py33
RESULT:
py33 cleaned up
[dhcp-40-8:~/doc/blog] (ruby-2.1.2) [ master ] ftweedal% rhc ssh py33
Connecting to 542a4b0f500446f46e000480.com ...

    *********************************************************************

    You are accessing a service that is for use only by authorized users.
    If you do not have authorization, discontinue use at once.
    Any use of the services is subject to the applicable terms of the
    agreement which can be found at:
    https://www.openshift.com/legal

    *********************************************************************

    Welcome to OpenShift shell

    This shell will assist you in managing OpenShift applications.

    !!! IMPORTANT !!! IMPORTANT !!! IMPORTANT !!!
    Shell access is quite powerful and it is possible for you to
    accidentally damage your application.  Proceed with care!
    If worse comes to worst, destroy your application with "rhc app delete"
    and recreate it
    !!! IMPORTANT !!! IMPORTANT !!! IMPORTANT !!!

    Type "help" for more info.


[py33-ftweedal.rhcloud.com 542a4b0f500446f46e000480]\> ls -la /tmp
total 0
[py33-ftweedal.rhcloud.com 542a4b0f500446f46e000480]\> touch /tmp/file
touch: cannot touch `/tmp/file': No such file or directory
[py33-ftweedal.rhcloud.com 542a4b0f500446f46e000480]\> mkdir /tmp/dir
mkdir: cannot create directory `/tmp/dir': No such file or directory
[py33-ftweedal.rhcloud.com 542a4b0f500446f46e000480]\> exit
Shared connection to py33-ftweedal.rhcloud.com closed.
[dhcp-40-8:~/doc/blog] (ruby-2.1.2) [ master ] ftweedal% 


---

Am I doing something wrong?

Comment 4 Fraser Tweedale 2014-09-30 07:13:28 UTC
Furthermore, I can report that after `rhc app tidy`:

[py33v2-ftweedal.rhcloud.com 542a56c64382ec5e160001d0]\> ls -ld /tmp
drwxrwx---. 0 542a56c64382ec5e160001d0 root 0 Sep 30 03:10 /tmp

Prior to tidy it was:

[py33v2-ftweedal.rhcloud.com 542a56c64382ec5e160001d0]\> ls -ld /tmp
drwxrwx---. 2 542a56c64382ec5e160001d0 root 4096 Sep 30 03:08 /tmp

So afaict, something is definitely awry.

Comment 5 Meng Bo 2014-10-13 06:56:29 UTC
@Fraser Tweedale 
You should reopen the bug if it still can be reproduced for you, or else no one will care about this.

Comment 6 Jhon Honce 2014-10-14 17:01:18 UTC
Please try again, the permissions to your gear's /tmp have been corrected.

Comment 7 openshift-github-bot 2014-10-16 02:46:53 UTC
Commits pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/ea91dfd908e7ac48cf433b9a68a77ef24156767c
Bug 1146750 - Ensure permissions on temporary directories

https://github.com/openshift/origin-server/commit/c804db94db505f3610e9fcd7c16f2ae2214b235e
Bug 1146750 - Ensure permissions on temporary directories

* fix code format

Comment 8 Qixuan Wang 2014-10-20 07:03:56 UTC
Tested on devenv_5247:

@Fraser Tweedale: I can reproduce your problem if I execute "rhc app tidy" and create directory/file under /tmp without terminating ssh.

@Jhon Honce: After "rhc app tidy", I reconnect the app and create directory/file under /tmp, it works well. Why "reconnect" is necessary? 

Please refer to the following results:

[py27s-d.dev.rhcloud.com 5444dc2fa822d05a92000003]\> mkdir /tmp/testdir
[py27s-d.dev.rhcloud.com 5444dc2fa822d05a92000003]\> touch /tmp/testfile
[py27s-d.dev.rhcloud.com 5444dc2fa822d05a92000003]\> ls -la /tmp
total 12
drwx-----T.  3 5444dc2fa822d05a92000003 root                     4096 Oct 20 06:42 .
dr-xr-xr-x. 26 root                     root                     4096 Oct 20 02:22 ..
-rw-r--r--.  1 5444dc2fa822d05a92000003 5444dc2fa822d05a92000003    0 Oct 20 05:56 haproxy_reload.lock
drwx------.  2 5444dc2fa822d05a92000003 5444dc2fa822d05a92000003 4096 Oct 20 06:42 testdir
-rw-------.  1 5444dc2fa822d05a92000003 5444dc2fa822d05a92000003    0 Oct 20 06:42 testfile
srwx------.  1 5444dc2fa822d05a92000003 5444dc2fa822d05a92000003    0 Oct 20 05:56 wsgi.7790.0.1.sock
[py27s-d.dev.rhcloud.com 5444dc2fa822d05a92000003]\> ls -la /tmp
total 0
[py27s-d.dev.rhcloud.com 5444dc2fa822d05a92000003]\> ls -ld /tmp
drwx-----T. 0 5444dc2fa822d05a92000003 root 0 Oct 20 06:43 /tmp
[py27s-d.dev.rhcloud.com 5444dc2fa822d05a92000003]\> mkdir /tmp/testdir2
mkdir: cannot create directory `/tmp/testdir2': No such file or directory
[py27s-d.dev.rhcloud.com 5444dc2fa822d05a92000003]\> touch /tmp/testfile2
touch: cannot touch `/tmp/testfile2': No such file or directory

[py27s-d.dev.rhcloud.com 5444dc2fa822d05a92000003]\> exit
exit
Connection to py27s-d.dev.rhcloud.com closed.
[root@localhost test]# rhc ssh py27s
/usr/local/rvm/gems/ruby-2.0.0-p481/gems/highline-1.6.21/lib/highline/system_extensions.rb:230: warning: Insecure world writable dir /usr/local/rvm/gems/ruby-2.0.0-p481@global in PATH, mode 040777
Connecting to 5444dc2fa822d05a92000003.rhcloud.com ...

    *********************************************************************

    You are accessing a service that is for use only by authorized users.
    If you do not have authorization, discontinue use at once.
    Any use of the services is subject to the applicable terms of the
    agreement which can be found at:
    https://www.openshift.com/legal

    *********************************************************************

    Welcome to OpenShift shell

    This shell will assist you in managing OpenShift applications.

    !!! IMPORTANT !!! IMPORTANT !!! IMPORTANT !!!
    Shell access is quite powerful and it is possible for you to
    accidentally damage your application.  Proceed with care!
    If worse comes to worst, destroy your application with "rhc app delete"
    and recreate it
    !!! IMPORTANT !!! IMPORTANT !!! IMPORTANT !!!

    Type "help" for more info.


[py27s-d.dev.rhcloud.com 5444dc2fa822d05a92000003]\> mkdir /tmp/testdir2
[py27s-d.dev.rhcloud.com 5444dc2fa822d05a92000003]\> touch /tmp/testfile2

Comment 9 Jhon Honce 2014-10-20 16:19:05 UTC
Fixed in https://github.com/openshift/origin-server/pull/5889

Node will no longer remove PI /tmp directory during tidy.

Comment 10 openshift-github-bot 2014-10-20 17:12:39 UTC
Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/3499753498406cde337f4e12f7c70a4f9943b752
Bug 1146750 - Do not remove PI tmp directory when tidying

Comment 11 Qixuan Wang 2014-10-21 03:08:44 UTC
Tested on devenv_5253, the bug has been fixed, thanks.

The testing results look as following,
[rb20s-d.dev.rhcloud.com 544603ca8b9e4daa28000001]\> touch /tmp/file1
[rb20s-d.dev.rhcloud.com 544603ca8b9e4daa28000001]\> mkdir /tmp/dir1
[rb20s-d.dev.rhcloud.com 544603ca8b9e4daa28000001]\> ls -la /tmp
total 12
drwx-----T.  3 544603ca8b9e4daa28000001 root                     4096 Oct 21 03:01 .
dr-xr-xr-x. 26 root                     root                     4096 Oct 21 02:44 ..
drwxr-xr-x.  3 544603ca8b9e4daa28000001 544603ca8b9e4daa28000001 4096 Oct 21 03:00 passenger

tidy the app on another terminal

[rb20s-d.dev.rhcloud.com 544603ca8b9e4daa28000001]\> mkdir /tmp/dir2
[rb20s-d.dev.rhcloud.com 544603ca8b9e4daa28000001]\> touch /tmp/file2


Note You need to log in before you can comment on or make changes to this bug.