Bug 989467 - Meet "Source Code repository could not be cloned: " error when creating app using git@ format source url on INT.
Meet "Source Code repository could not be cloned: " error when creating app u...
Status: CLOSED UPSTREAM
Product: OpenShift Online
Classification: Red Hat
Component: Containers (Show other bugs)
2.x
Unspecified Unspecified
medium Severity medium
: ---
: ---
Assigned To: Hiro Asari
libra bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-07-29 06:23 EDT by Yujie Zhang
Modified: 2015-05-14 19:25 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-07-31 14:32:34 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
clone error (123.98 KB, image/png)
2013-07-30 01:39 EDT, Yujie Zhang
no flags Details

  None (edit)
Description Yujie Zhang 2013-07-29 06:23:27 EDT
Description of problem:

Tried to create spring application using git format url : git@github.com:openshift/spring-eap6-quickstart.git, but met Source Code repository could not be cloned: 'git@github.com:openshift/spring-eap6-quickstart.git'. Please verify the repository is correct and contact support. 

Same result when creating using latest rhc.

This issue only exists on INT, devenv_3572 can create app successfully.

Version-Release number of selected component (if applicable):

INT(devenv_3569)

rhc 1.12.3

How reproducible:
always

Steps to Reproduce:
1.Try to create app using url "git@github.com:openshift/spring-eap6-quickstart.git" from website
2.rhc app create -a sprint -t jbosseap-6.0 --from-code git@github.com:openshift/spring-eap6-quickstart.git --server int.openshift.redhat.com 
3.

Actual results:

Met "Source Code repository could not be cloned:" error, rhc result is like following: 

[root@dhcp-8-252 Downloads]# rhc app create -a sprint -t jbosseap-6.0 --from-code git@github.com:openshift/spring-eap6-quickstart.git --server int.openshift.redhat.com -l yujzhang+int999@redhat.com
Your authorization token has expired. Please sign in now to continue.
Password: ******

Application Options
-------------------
  Namespace:   gasdf
  Cartridges:  jbosseap-6.0 (addtl. costs may apply)
  Source Code: git@github.com:openshift/spring-eap6-quickstart.git
  Gear Size:   default
  Scaling:     no

Creating application 'sprint' ... 
Source Code repository could not be cloned:
'git@github.com:openshift/spring-eap6-quickstart.git'.  Please verify the
repository is correct and contact support.


Expected results:

Should be able to create app using git format url successfully.

Additional info:
Comment 1 Hiro Asari 2013-07-29 11:10:13 EDT
github is having API issues resulting from a large-scale DDoS attack. Wait for a while, and try again.

https://status.github.com/messages
Comment 2 Yujie Zhang 2013-07-30 01:38:38 EDT
The github is normal now, I tried on INT(devenv_3578), but still got the same issue, details see in attachment please.
Comment 3 Yujie Zhang 2013-07-30 01:39:13 EDT
Created attachment 780333 [details]
clone error
Comment 4 Yujie Zhang 2013-07-30 01:53:19 EDT
It works correctly with git@ format on devenv_3580 but does not work on INT. For INT, using format git:// can create application successfully.
Comment 5 Hiro Asari 2013-07-30 16:40:51 EDT
On INT, the error manifests as something like this:

July 29 05:52:41 INFO oo_spawn running /sbin/runuser -s /bin/sh 51f63b3e6cec0ee3120001a8 -c "exec /usr/bin/runcon 'unconfined_u:system_r:openshift_t:s0:c6,c346' /bin/sh -c \"
set -xe;                                                                               
shopt -s dotglob;                                                                      
if [ "$(find objects -type f 2>/dev/null | wc -l)" -eq "0" ]; then                     
  exit 0;                                                                              
fi                                                                                     
git archive --format=tar HEAD | (cd /var/lib/openshift/51f63b3e6cec0ee3120001a8/app-root/runtime/repo && tar --warning=no-timestamp -xf -); 
\"": {:unsetenv_others=>true, :close_others=>true, :in=>"/dev/null", :chdir=>"/var/lib/openshift/51f63b3e6cec0ee3120001a8/git/treasuredata.git", :out=>#<IO:fd 14>, :err=>#<IO:fd 11>}
July 29 05:52:42 INFO oo_spawn buffer(6/) + git clone --bare --no-hardlinks git@github.com:openshift/spring-eap6-quickstart.git sprint.git
                                                                                       
July 29 05:52:42 INFO oo_spawn buffer(12/) Cloning into bare repository 'sprint.git'...
                                                                                       
July 29 05:52:42 INFO oo_spawn buffer(6/) Host key verification failed.^M       
                                                                                       
July 29 05:52:42 INFO oo_spawn buffer(6/) fatal: Could not read from remote repository.
                                                                                       
Please make sure you have the correct access rights                                    
and the repository exists.
Comment 6 Hiro Asari 2013-07-30 22:37:50 EDT
See https://github.com/openshift/origin-server/pull/3239
Comment 7 Hiro Asari 2013-07-31 12:38:21 EDT
The above PR will put a message to use a non-SSH URL in this situation.

If you remove /root/.ssh/id_rsa on devenv, the application creation via SSH URL will fail. This is pretty reasonable, since the public key for this key was probably uploaded to Github, and the root user on devenv has read access to the quickstarts which have been tested so far.

If you try cloning from a SSH URL to which the devenv root user does not have read access, the application creation will similarly fail:

$ bx bin/rhc app create foobar ruby-1.9 mysql-5.1 --from-code git@bitbucket.org:wikimatze/rails-sample-app.git
Application Options
-------------------
  Namespace:   fooooooooooo
  Cartridges:  ruby-1.9, mysql-5.1
  Source Code: git@bitbucket.org:wikimatze/rails-sample-app.git
  Gear Size:   default
  Scaling:     no

Creating application 'foobar' ... 
Source Code repository could not be cloned: 'git@bitbucket.org:wikimatze/rails-sample-app.git'.  Please verify the repository is correct and contact support.


Has this ever work on INT or PRD?

Setting up the SSH keys on INT and PRD images *should* fix the problem, but it is just for Github, which may not be sufficient for the reason that I indicated above. If the user wants to pull from bitbucket, say, it would fail.

In general, we will need a mechanism to pull code from any Git URL given sufficient authorization from the user. But that is going to be a little bigger than a simple bugzilla.
Comment 8 Hiro Asari 2013-07-31 12:57:45 EDT
"Has this ever work on INT or PRD?" => "Did cloning a repo from Github ever work on INT or PRD?"
Comment 9 openshift-github-bot 2013-07-31 13:33:30 EDT
Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/73db35011afb3b9430b34e7143c1d5909a98996d
Bug 989467

This commit does *not* fix the problem described in the Bugzilla.
Rather, when git clone fails when using SSH, suggest using a different
URL scheme so that the authentication is less of a problem.

Here, the SSH URL detection is purely formal.
The passed URL is deemed SSH if one of the following condition is true:
  1. it starts with 'ssh:'
  2. it contains '@' and not '//' (consider http://user:pass@example.com/foo.git)
Comment 10 Hiro Asari 2013-07-31 14:32:34 EDT
Trello card to address the larger issue: https://trello.com/c/z3FCdXTq/206-devise-a-means-to-clone-a-git-repository-over-ssh
Comment 11 Yujie Zhang 2013-07-31 23:03:45 EDT
(In reply to Hiro Asari from comment #8)
It worked before when using git@github.com:openshift/spring-eap6-quickstart.git, and it still works on devenv_3597, but does not work on INT.
Comment 12 Hiro Asari 2013-08-01 08:56:16 EDT
(In reply to Yujie Zhang from comment #11)
> (In reply to Hiro Asari from comment #8)
> It worked before when using
> git@github.com:openshift/spring-eap6-quickstart.git, and it still works on
> devenv_3597, but does not work on INT.

To close the loop on this...

I suspect that the reason for this 'regression' is that the ops changed the SSH key on INT. The fact that it used to work is a coincidence rather than design, in my opinion. Since cloning over SSH is not reliable in the first place, we should deal with this in a more general fashion. This is precisely what the Trello card does.

Note You need to log in before you can comment on or make changes to this bug.