Bug 982436 - Unable to use any functionality for existing applications i.e. 'rhc apps' or 'rhc create-app' etc.
Unable to use any functionality for existing applications i.e. 'rhc apps' or ...
Status: CLOSED CURRENTRELEASE
Product: OpenShift Online
Classification: Red Hat
Component: Pod (Show other bugs)
1.x
x86_64 Mac OS
unspecified Severity urgent
: ---
: ---
Assigned To: Rajat Chopra
libra bugs
: UpcomingRelease
: 982658 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-07-08 23:43 EDT by Edward Archibald
Modified: 2013-07-22 11:22 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-07-22 11:22:41 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Edward Archibald 2013-07-08 23:43:37 EDT
Description of problem:
Sometime around 11 AM PDT, Mon. Jul 8, 2013, I was unable to interact with OpenShift, wither via command line or via the UI, for operations like:

rhc apps
rhc port-forward 
rhc app delete
rhc create-app
etc.

For any of these command, I consistently get the same message:

The server did not respond correctly. This may be an issue with the server configuration or with your connection to the server
(such as a Web proxy or firewall). Please verify that you can access the OpenShift server
https://openshift.redhat.com/broker/rest/domains/continuent/applications/tungstencluster

When I check the liveness of that URL, I get a username/password challenge after which (I provide a valid username/password) I get this message:

We're sorry, but something went wrong.

We've been notified about this issue and we'll take a look at it shortly.

Clicking on the 'My Applications' or 'My Apps' buttons on the UI yield the message:

We appear to be having technical difficulties
You can try refreshing the page, if the problem is temporary.

This has been going on for about 10 hours at this point.  
 

Version-Release number of selected component (if applicable):
I am running against the current production online broker etc.  I am running rhc 1.9.6 command line tools.

My account ID edward.archibald@continuent.com

Steps to Reproduce:
1. Establish a session with OpenShift online via rhc.  I logged out and then back in just in case something was hosed in that path.
2. rhc apps

Actual results:
The server did not respond correctly. This may be an issue with the server configuration or with your connection to the server
(such as a Web proxy or firewall). Please verify that you can access the OpenShift server
https://openshift.redhat.com/broker/rest/domains/continuent/applications/tungstencluster


Expected results:
A listing of my current apps


Additional info:
I am able to log into the gear on which my app/cartridge is running if I log in directly using the username/host i.e. ssh 51da0957500446f29700005b@tungstencluster-continuent.rhcloud.com but if I try to use anything that looks up apps, like 'rhc ssh tungstencluster', it fails.
Comment 1 Riccardo Vianello 2013-07-09 06:55:06 EDT
I think I'm experiencing the same issue. The reference reported by the web UI when accessing 'my apps' is the following: #335c5e33ffa8761f79d90922bd070829
Comment 2 dz0ny 2013-07-09 14:37:56 EDT
I also have the same problem. Reference #119a250574f10436db8f3acc7cfaf166
Comment 3 Edward Archibald 2013-07-09 14:40:49 EDT
I am using a downloadable cartridge in this app - it's a cartridge that I am developing.  The cartridge git is at:  narmitag/openshift-origin-cartridge-tungsten 

The cartridge work has been proceeding well up until I hit this issue - for about a week.
Comment 4 Edward Archibald 2013-07-09 14:46:40 EDT
Here's the exact command I used to create the app with the downloadable cartridge:

rhc create-app tungstencluster  php-5.3  http://cartreflect-claytondev.rhcloud.com/reflect?github=narmitag/openshift-origin-cartridge-tungsten --gear medium
Comment 5 Abhishek Gupta 2013-07-09 15:49:13 EDT
We have identified the problem and are working to fix the issue as well as the broken applications. The fix should not take more than a few hours. Will update this bug when the fix is complete.
Comment 6 Rajat Chopra 2013-07-09 18:55:13 EDT
There was a bug with a maintenance script that affected all applications with downloaded cartridges. The erring maintenance was run around 4pm July 8. 

All affected applications created before 2013-07-08 00:00:00EDT have been fixed.

If there was an application created (with downloadable cartridges) between 2013-07-08 00:00:00EDT and 2013-07-08 16:00:00 EDT, then it may still be broken. We are working on fixing them too.

Apologies for any hassle that this may have caused.
Comment 7 Rajat Chopra 2013-07-09 21:30:39 EDT
*** Bug 982658 has been marked as a duplicate of this bug. ***
Comment 8 Abhishek Gupta 2013-07-10 18:43:51 EDT
Assigning it to Rajat as he is working on identifying and fixing any remaining impacted applications.
Comment 9 Riccardo Vianello 2013-07-11 07:37:44 EDT
Thank you for working on this issue. The problematic application for my account is most likely to be 'mycart'. It's just a test application that doesn't currently provide any functionality, so please feel free to simply erase it if this may help a quicker resolution.
Comment 10 Rajat Chopra 2013-07-11 15:56:00 EDT
All known applications have been fixed. Please report if anything is still not working.
Comment 11 Riccardo Vianello 2013-07-12 02:00:56 EDT
My account (ID is riccardo dot vianello at gmail dot com) is still unusable. Latest reference displayed by the web UI is #50d9909ed3fc229803fa2b24327e3103.
Comment 12 Xiaoli Tian 2013-07-15 04:30:39 EDT
Move it to assigned to see if more fixing is needed according to comment 11 or if more migration is needed.
Comment 13 Rajat Chopra 2013-07-16 18:52:03 EDT
The erring app was deleted with user's permission (comment#9). This application apparently had more than one downloaded cartridges, and the fix reported in comment#10 missed that complication.

Hope everything is working now.
Comment 14 Riccardo Vianello 2013-07-17 02:02:19 EDT
Confirmed, my account is functional again. Thank you very much for your help.
Comment 15 Xiaoli Tian 2013-07-17 06:26:04 EDT
Move it to verified since the account is fixed according to comment 14

Note You need to log in before you can comment on or make changes to this bug.