Bug 1018375 - app creation with custom manifest will fail if the http_url is hosted in github due to github redirecting all http requests to https
app creation with custom manifest will fail if the http_url is hosted in gith...
Status: CLOSED WONTFIX
Product: OpenShift Online
Classification: Red Hat
Component: Containers (Show other bugs)
2.x
Unspecified Unspecified
unspecified Severity low
: ---
: ---
Assigned To: Maciej Szulik
libra bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-10-11 15:40 EDT by Peter Ruan
Modified: 2015-05-14 19:30 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-07-22 04:52:24 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
manifest file used (2.61 KB, text/x-vhdl)
2013-11-07 21:51 EST, Qiushui Zhang
no flags Details

  None (edit)
Description Peter Ruan 2013-10-11 15:40:12 EDT
Description of problem:
  app creation with custom manifest will fail if the http_url is hosted in github due to github redirecting all http requests to https

Version-Release number of selected component (if applicable):
current

How reproducible:
always.

Steps to Reproduce:
1. rhc app create fee1kj https://raw.github.com/openshift-qe/cucushifting/master/manifests/phpmaster/manifest_http_bad.yml
2.
3.

Actual results:
[11:34:59] INFO> Shell Command: rhc app create fee1kj https://raw.github.com/openshift-qe/cucushifting/master/manifests/phpmaster/manifest_http.yml -l pruan@redhat.com -p 'vostok08' --insecure --server openshift.redhat.com
      The cartridge 'https://raw.github.com/openshift-qe/cucushifting/master/manifests/phpmaster/manifest_http.yml' will be downloaded and installed
      
      Application Options
      -------------------
        Domain:     inorqu
        Cartridges: https://raw.github.com/openshift-qe/cucushifting/master/manifests/phpmaster/manifest_http.yml
        Gear Size:  default
        Scaling:    no
      
      Creating application 'fee1kj' ... Unexpected error: redirection forbidden: http://raw.github.com/openshift-qe/cucushifting/master/manifests/phpmaster/phpv2cart.tar -> https://raw.github.com/openshift-qe/cucushifting/master/manifests/phpmaster/phpv2cart.tar
      [11:35:40] INFO> Exit Status: 1


Expected results:


Additional info:
Comment 1 Lili Nader 2013-11-05 18:02:55 EST
This error is occurring on node.  Snippet from mcollective log file

E, [2013-11-05T18:00:36.590514 #1193] ERROR -- : openshift.rb:312:in `rescue in with_container_from_args' CLIENT_ERROR: Unexpected error: redirection forbidden: http://raw.github.com/openshift-qe/cucushifting/master/manifests/phpmaster/phpv2cart.tar -> https://raw.github.com/openshift-qe/cucushifting/master/manifests/phpmaster/phpv2cart.tar
E, [2013-11-05T18:00:36.590694 #1193] ERROR -- : openshift.rb:313:in `rescue in with_container_from_args' /opt/rh/ruby193/root/usr/share/ruby/open-uri.rb:216:in `open_loop'
/opt/rh/ruby193/root/usr/share/ruby/open-uri.rb:146:in `open_uri'
/opt/rh/ruby193/root/usr/share/ruby/open-uri.rb:677:in `open'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.17.0/lib/openshift-origin-node/model/cartridge_repository.rb:515:in `block in uri_copy'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-common-1.17.0/lib/openshift-origin-common/utils/file_needs_sync.rb:38:in `block in open'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-common-1.17.0/lib/openshift-origin-common/utils/file_needs_sync.rb:36:in `open'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-common-1.17.0/lib/openshift-origin-common/utils/file_needs_sync.rb:36:in `open'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.17.0/lib/openshift-origin-node/model/cartridge_repository.rb:514:in `uri_copy'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.17.0/lib/openshift-origin-node/model/cartridge_repository.rb:474:in `instantiate_cartridge'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.17.0/lib/openshift-origin-node/model/v2_cart_model.rb:500:in `create_cartridge_directory'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.17.0/lib/openshift-origin-node/model/v2_cart_model.rb:257:in `block in configure'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.17.0/lib/openshift-origin-node/utils/cgroups.rb:126:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.17.0/lib/openshift-origin-node/utils/cgroups.rb:126:in `apply_profile'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.17.0/lib/openshift-origin-node/utils/cgroups.rb:44:in `block (2 levels) in <class:Cgroups>'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.17.0/lib/openshift-origin-node/model/v2_cart_model.rb:256:in `configure'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.17.0/lib/openshift-origin-node/model/application_container_ext/cartridge_actions.rb:27:in `configure'
/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:863:in `block in oo_configure'
/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:301:in `with_container_from_args'
/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:862:in `oo_configure'
/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:139:in `execute_action'
/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:104:in `cartridge_do_action'
/opt/rh/ruby193/root/usr/share/ruby/mcollective/rpc/agent.rb:86:in `handlemsg'
/opt/rh/ruby193/root/usr/share/ruby/mcollective/agents.rb:126:in `block (2 levels) in dispatch'
/opt/rh/ruby193/root/usr/share/ruby/timeout.rb:69:in `timeout'
Comment 2 Paul Morie 2013-11-06 14:19:16 EST
I could not recreate this in a devenv.  Is it still an issue?
Comment 3 Qiushui Zhang 2013-11-07 21:50:01 EST
Tested on devenv_4003.
Prepare manifest.yml with a http link instead of https link, like the following:

Source-Url: http://github.com/qiushui/php/raw/master/php.tar


The app creation will fail:
openshift@openshift-ubuntu:~/tmp$ rhc app create php3 https://raw.github.com/qiushui/php/master/metadata/manifest.yml
The cartridge 'https://raw.github.com/qiushui/php/master/metadata/manifest.yml' will be downloaded and installed

Application Options
-------------------
  Domain:     qiuzhang
  Cartridges: https://raw.github.com/qiushui/php/master/metadata/manifest.yml
  Gear Size:  default
  Scaling:    no

Creating application 'php3' ... 
Unexpected error: redirection forbidden: http://github.com/qiushui/php/raw/master/php.tar ->
https://github.com/qiushui/php/raw/master/php.tar


If changing "http" to "https" in manifest.yml  "Source-Url" part, the creation will success.

Please refer to the attachment for the manifest file used.

Mark the bug as failed since it still fails to create app with "http" link, which is similar to the bug reporter's description.
Comment 4 Qiushui Zhang 2013-11-07 21:51:15 EST
Created attachment 821421 [details]
manifest file used
Comment 5 Chris Ryan 2014-07-15 16:56:27 EDT
This is still an issue as of devenv_4967. 

POST https://$OPENSHIFT_BROKER/broker/rest/domain/uoqixg/applications {"name":"app10","cartridges":[{"url":"https://raw.githubusercontent.com/openshift-qe/cartridge_manifests_repo/master/tc266481/http_zip_manifest.yml"}]}

ERROR> {"api_version":1.7,"data":null,"messages":[{"exit_code":1,"field":null,"index":null,"severity":"error","text":"Unexpected error: redirection forbidden: http://github.com/openshift-qe/perlv2cart/raw/master/perlv2cart.zip -> https://github.com/openshift-qe/perlv2cart/raw/master/perlv2cart.zip\n"}],"status":"unprocessable_entity","supported_api_versions":[1.0,1.1,1.2,1.3,1.4,1.5,1.6,1.7],"type":null,"version":"1.7"}
Comment 6 Ben Parees 2014-07-15 17:13:30 EDT
Redirects when retrieving the source_url are explicitly forbidden by the code, the test case should be amended to host the source_url archive at a url that does not involve a redirect (both http and https urls should be tested, however).
Comment 7 Aleksandar Kostadinov 2014-07-15 17:33:38 EDT
Ben, what is the rational behind forbidding redirects? Somebody suggested security. Are they less secure than using plain http? Who made that decision? Was RH security team involved?

In my opinion redirects are a standard mechanism to keep content available through particular URL and they are not a greater risk than especially used over https.
Comment 8 Ben Parees 2014-07-15 17:47:07 EDT
This is actually a restriction of the ruby open-uri library that is being used:
https://bugs.ruby-lang.org/issues/859
Comment 9 Chris Ryan 2014-07-15 17:49:21 EDT
According to that link, it seems like a new gem was spun-off to address that very issue: https://github.com/jaimeiniesta/open_uri_redirections
Comment 10 Ben Parees 2014-07-16 16:59:35 EDT
Maciejz, can you take a look at reworking the call in question to use the library in comment 9?

The call is in cartridge_repostory.rb:573
Comment 11 Maciej Szulik 2014-07-22 04:52:24 EDT
After discussion with Dan, Michal and Ben we've decided not to fix this issue. The reason for this is the SCL ruby we're using does not contain open_uri_redirections gem, so we'd have to include that ourselves, which creates other concers namely security, maintenance & licencing.

Note You need to log in before you can comment on or make changes to this bug.