Bug 851390

Summary:	old scalable app cannot push build successful after server upgraded on stage
Product:	OKD	Reporter:	Meng Bo <bmeng>
Component:	Containers	Assignee:	Rob Millner <rmillner>
Status:	CLOSED CURRENTRELEASE	QA Contact:	libra bugs <libra-bugs>
Severity:	low	Docs Contact:
Priority:	medium
Version:	2.x	CC:	jhou, mfisher, mmcgrath, xtian
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2012-11-06 18:50:22 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Meng Bo 2012-08-24 03:39:24 UTC

Description of problem:
Old scalable app exist on stage, after server upgraded and migrated, try to change the repo and push the build via git push, will fail with exit code 1.

=============
<--->
remote: Warning: Identity file /var/lib/stickshift/25fa25c7cf72430b8a1c574dc7cd1947//haproxy-1.4/.ssh/haproxy_id_rsa not accessible: Permission denied.
remote: rsync: connection unexpectedly closed (0 bytes received so far) [sender]
remote: rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.6]
remote: Exit code: 1
<--->

Version-Release number of selected component (if applicable):
stage_2.0.16

How reproducible:
always

Steps to Reproduce:
1.Old scalable app exist on server
2.Do server upgrade and migrate
3.Change the app repo and git push
  
Actual results:
Build failed with exit code 1.

Expected results:
Push build should not have any problem. 

Additional info:
This issue does not exist on INT after server upgrade.
This issue does not exist for the new created scalable app on STG.

Comment 1 Xiaoli Tian 2012-08-24 05:15:04 UTC

Login to the haproxy gear and ssh to its another gear in this scaling app, failed with the same error :

 [phps-stgdm4.stg.rhcloud.com conf]\> ssh b168350d05-stgdm4.stg.rhcloud.com
 Warning: Identity file /var/lib/stickshift/0aeb5a9836294074bbb6017834ace16b//haproxy-1.4/.ssh/haproxy_id_rsa not accessible: Permission denied.

Comment 2 Rob Millner 2012-08-24 18:50:24 UTC

Where did this app come from?  Was it always on stg or was it unpacked from a backup?

Comment 3 Rob Millner 2012-08-24 18:55:30 UTC

Setting low priority.  I believe this is due to the SELinux hiccup last sprint which was resolved except for this app.

Comment 4 Rob Millner 2012-08-24 20:08:05 UTC

Added comment to the release ticket to run rhc-restorecon by hand on every exec node except the C9 nodes.

Comment 5 Rob Millner 2012-08-24 20:14:26 UTC

The rhc-restorecon command was run on the stg node containing the bad application and it fixed the SELinux label.

Comment 6 Meng Bo 2012-08-27 06:31:27 UTC

Checked on latest stage env, for my existing applications, some of them have been fixed. But some of them still can get the same error. 
50% of them have been fixed.

I list my failed apps info.
account: qgong
apps:
perl1s-qgongstg.stg.rhcloud.com
no1s-qgongstg.stg.rhcloud.com
py1s-qgongstg.stg.rhcloud.com

account: bmeng+1
apps:
jbas1s-bmeng1stg.stg.rhcloud.com

Comment 7 Mike McGrath 2012-10-10 18:52:51 UTC

We think this is solved now, moving to on_qa.  Please let us nkow if it's still happening.

  - Bug Triage Meeting

Comment 8 Jianwei Hou 2012-10-11 02:20:57 UTC

Verified on stage

After upgrade on stage, git push an old existing scaling app was successful.