Bug 987876

We weren't able to reproduce the issue. Could you please retry?

Assigned this bug back since it is reproducible in INT(devenv_3561). Take a scalig php-5.3 app as an example, after scaling it up and showing its gear groups, both its local gear and scaled-up gears were displayed:

"gears": [
                {
                    "id": "51f1e37a6cec0e43bf000001", 
                    "ssh_url": "ssh://51f1e37a6cec0e43bf000001.rhcloud.com", 
                    "state": "started"
                }, 
                {
                    "id": "51f1e4c803ef647e3e000002", 
                    "ssh_url": "ssh://51f1e4c803ef647e3e000002.rhcloud.com", 
                    "state": "new"
                }
            ], 

SSH into both gears and check the repos in both gears.

Results:
In the app's local gear (i.e., 51f1e37a6cec0e43bf000001.rhcloud.com), the dirs are complete:

[sphp-zwint.int.rhcloud.com 51f1e37a6cec0e43bf000001]\> ls -l
total 16
drwxr-xr-x.  4 root                     51f1e37a6cec0e43bf000001 4096 Jul 25 22:48 app-root
drwxr-xr-x.  3 root                     root                     4096 Jul 25 22:48 git
drwxr-xr-x. 12 51f1e37a6cec0e43bf000001 51f1e37a6cec0e43bf000001 4096 Jul 25 22:49 haproxy
drwxr-xr-x. 14 51f1e37a6cec0e43bf000001 51f1e37a6cec0e43bf000001 4096 Jul 25 22:48 php

However, in its scaled-up gear (51f1e4c803ef647e3e000002.rhcloud.com), only app-root dir was displayed.

[51f1e4c803ef647e3e000002-zwint.int.rhcloud.com 51f1e4c803ef647e3e000002]\> ls -l
total 4
drwxr-xr-x. 4 root 51f1e4c803ef647e3e000002 4096 Jul 25 22:54 app-root

Moreover, when showing the repo dir under the app-root dir, the content was empty:

[51f1e4c803ef647e3e000002-zwint.int.rhcloud.com 51f1e4c803ef647e3e000002]\> ls app-root/repo/
[51f1e4c803ef647e3e000002-zwint.int.rhcloud.com 51f1e4c803ef647e3e000002]\>

I have reserved the scaling php-5.3 app in INT(devenv_3561) for your further investigation, and the detail of the app is as below:

        "aliases": [], 
        "app_url": "http://sphp-zwint.int.rhcloud.com/", 
        "build_job_url": null, 
        "building_app": null, 
        "building_with": null, 
        "creation_time": "2013-07-26T02:48:26Z", 
        "domain_id": "zwint", 
        "embedded": {
            "haproxy-1.4": {}
        }, 
        "framework": "php-5.3", 
        "gear_count": 2, 
        "gear_profile": "small", 
        "git_url": "ssh://51f1e37a6cec0e43bf000001.rhcloud.com/~/git/sphp.git/", 
        "health_check_path": "health_check.php", 
        "id": "51f1e37a6cec0e43bf000001", 
        "initial_git_url": null,

I confirm that this bug only happens if we scale up apps with REST API, there is no such problem when scaling up apps with "rhc cartridge scale". Therefore, I have modified the summary of this bug accordingly.

More info of this bug what would be helpful:

After scaling up the app with REST API, we can confirm the app has two gears:

[zhe@fedora sphp1]$ rhc app-show --gears -a sphp
ID                       State   Cartridges          Size  SSH URL
------------------------ ------- ------------------- ----- -----------------------------------------------------------------------
51f1e37a6cec0e43bf000001 started php-5.3 haproxy-1.4 small 51f1e37a6cec0e43bf000001.rhcloud.com
51f1e4c803ef647e3e000002 new     php-5.3 haproxy-1.4 small 51f1e4c803ef647e3e000002.rhcloud.com

However, the status of the scaled-up gear is always "new"[1], and the scaled-up gear is not accessible via its URL (you could try with the URL of the scaled-up gear in Comment #2).

[1]https://bugzilla.redhat.com/show_bug.cgi?id=987777

On both INT(devenv_3561) and devenv_3564, another problem caused by scaling up apps via REST API is that, it will fail to control apps after scaling it up, with the following error (Please refer to the attachment for more info):

[zhe@fedora run106]$ rhc app stop sphp1
Unable to complete the requested operation due to: Failed to correctly execute all parallel operations.
Reference ID: 5479cc0db2eab26bea94d827d94d43cd

Steps to reproduce:
1) create a sclaing app
2) scaling it up with REST API
3) try to control this app, for example, stop it via RHC

Looks like scaling up an app with REST API is not completed, making the app is kind of "locked". If this is true, the reasoning could explain this bug, Bug 98777, and the problem described in this comment.

Created attachment 778692 [details]
Corresponding logs of failing to control the scaled app

Sorry, I wanted to refer to Bug 987777 in Comment #6.

I confirm that before controlling the app in Comment #6, the broker returned the message of successfully scaling up the app.

    "messages": [
        {
            "exit_code": 0, 
            "field": null, 
            "severity": "info", 
            "text": "Application sphp1 has scaled up"
        }
    ],

The bug has been fixed in devenv_3574.

Steps:
1) create a scaling app
2) scale it up with REST API
3) show the app's gear groups
4) SSH into the scaled-up gear and checking the repo in the scaled-up gear
5) control this app, for example, stop, start, restart, reload

Results:
The scaling-up in Step 2 succeeded, and when showing the app's gear groups, the status of all gears is "started".

            "gears": [
                {
                    "id": "51f62a10fe91e30a98000001", 
                    "ssh_url": "ssh://51f62a10fe91e30a98000001.rhcloud.com", 
                    "state": "started"
                }, 
                {
                    "id": "51f62b4afe91e30a98000004", 
                    "ssh_url": "ssh://51f62b4afe91e30a98000004.rhcloud.com", 
                    "state": "started"
                }
            ], 

Moreover, the copy of the app's repo was synced to its scaled-up gear as well:

[51f62b4afe91e30a98000004-dev3574tst.dev.rhcloud.com 51f62b4afe91e30a98000004]\> ls -l
total 12
drwxr-xr-x.  4 root                     51f62b4afe91e30a98000004 4096 Jul 29 04:43 app-root
drwxr-xr-x.  3 root                     root                     4096 Jul 29 04:44 git
drwxr-xr-x. 16 51f62b4afe91e30a98000004 51f62b4afe91e30a98000004 4096 Jul 29 04:44 python

and, there was no problem to control this app.

Could you please move the bug to ON_QA, and I will verify this bug literally then.

Thanks,
Zhe Wang

Fixed with rev#0bc92a9d4b75514bf667eacdf356581b055d6b6c

Move the bug to VERIFIED referring to Comment #10.