Bug 1358844 - "Loading from Red Hat Customer Portal" hangs
Summary: "Loading from Red Hat Customer Portal" hangs
Keywords:
Status: CLOSED DUPLICATE of bug 1371616
Alias: None
Product: Red Hat Quickstart Cloud Installer
Classification: Red Hat
Component: Installation - RHELOSP
Version: 1.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: ---
Assignee: Erik Nelson
QA Contact: Dave Johnson
Dan Macpherson
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-07-21 15:18 UTC by Thom Carlin
Modified: 2016-08-30 20:06 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-08-30 20:06:34 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1358813 0 unspecified CLOSED Clicking on satellite navigation menu while on any wizard page takes you to product selection page 2021-02-22 00:41:40 UTC

Internal Links: 1358813

Description Thom Carlin 2016-07-21 15:18:36 UTC
Description of problem:

During RHOSP deployment, hangs at "Add Subscriptions"

Version-Release number of selected component (if applicable):

QCI-1.2-RHEL-7-20160720.t.0
QCIOOO-8.0-RHEL-7-20160719.t.0

How reproducible:

Uncertain

Steps to Reproduce:
1. Install and configure QCI and TripleO
2. Deploy RHOSP

Actual results:

Hang at "Loading from Red Hat Customer Portal"

Expected results:

No hangs
Successful deployment

Additional info:

More soon.  Others in our group have gotten past this point

Comment 1 Thom Carlin 2016-07-21 15:21:25 UTC
Excerpt from /var/log/foreman/production.log:
2016-07-21 07:04:10 [app] [W] fusor_server did not find 'Katello.config.logging.ignored_paths' available, skipping silence of logs for '["api/v21/foreman_tasks", "fusor/api/v21/unlogged"]'
2016-07-21 07:04:41 [app] [W] Foreman anonymous admin does not exist, skipping content host migration.
2016-07-21 07:04:46 [app] [W] Recreating the whole DB cache
2016-07-21 07:04:46 [foreman-tasks/dynflow] [I] start terminating delayed_executor...
2016-07-21 07:04:46 [foreman-tasks/dynflow] [I] start terminating throttle_limiter...
2016-07-21 07:04:46 [foreman-tasks/dynflow] [I] start terminating executor...
2016-07-21 07:04:46 [foreman-tasks/dynflow] [I] shutting down Core ...
2016-07-21 07:04:46 [foreman-tasks/dynflow] [E] ... core terminated.
2016-07-21 07:04:46 [foreman-tasks/dynflow] [I] start terminating executor dispatcher...
2016-07-21 07:04:46 [foreman-tasks/dynflow] [I] start terminating client dispatcher...

over..and...over

Comment 2 Thom Carlin 2016-07-21 15:28:17 UTC
javascript console shows:
Transitioned into 'deployment.subscriptions.select-subscriptions'

more from production.log:
2016-07-21 11:08:54 [app] [I] Started PUT "/fusor/api/v21/deployments/1" for 10.10.116.69 at 2016-07-21 11:08:54 -0400
2016-07-21 11:08:54 [app] [I] Processing by Fusor::Api::V21::DeploymentsController#update as JSON
2016-07-21 11:08:54 [app] [I]   Parameters: {"deployment"=>{"name"=>"<<deployment name>>", "label"=>"<<deployment name>>", "description"=>"", "deploy_rhev"=>false, "deploy_cfme"=>false, "deploy_openstack"=>true, "deploy_openshift"=>false, "is_disconnected"=>false, "has_content_error"=>false, "rhev_is_self_hosted"=>false, "rhev_engine_admin_password"=>"[FILTERED]", "rhev_data_center_name"=>"Default", "rhev_cluster_name"=>"Default", "rhev_storage_name"=>"my_storage", "rhev_storage_type"=>"NFS", "rhev_storage_address"=>nil, "rhev_cpu_type"=>nil, "rhev_share_path"=>nil, "cfme_install_loc"=>nil, "rhev_root_password"=>"[FILTERED]", "cfme_root_password"=>"[FILTERED]", "cfme_admin_password"=>"[FILTERED]", "cfme_db_password"=>"[FILTERED]", "foreman_task_uuid"=>nil, "upstream_consumer_uuid"=>"<<consumer_uuid>>", "upstream_consumer_name"=>"<<sma_name>>", "rhev_export_domain_name"=>"my_export", "rhev_export_domain_address"=>nil, "rhev_export_domain_path"=>nil, "hosted_storage_name"=>"my_hosted_storage", "hosted_storage_address"=>nil, "hosted_storage_path"=>nil, "rhev_local_storage_path"=>nil, "host_naming_scheme"=>"Freeform", "custom_preprend_name"=>nil, "enable_access_insights"=>false, "cfme_address"=>nil, "cfme_hostname"=>nil, "cdn_url"=>nil, "manifest_file"=>nil, "openshift_install_loc"=>nil, "openshift_number_master_nodes"=>nil, "openshift_number_worker_nodes"=>nil, "openshift_storage_size"=>nil, "openshift_username"=>nil, "openshift_user_password"=>"[FILTERED]", "openshift_root_password"=>"[FILTERED]", "openshift_master_vcpu"=>nil, "openshift_master_ram"=>nil, "openshift_master_disk"=>nil, "openshift_node_vcpu"=>nil, "openshift_node_ram"=>nil, "openshift_node_disk"=>nil, "openshift_available_vcpu"=>nil, "openshift_available_ram"=>nil, "openshift_available_disk"=>nil, "openshift_storage_type"=>nil, "openshift_storage_host"=>nil, "openshift_export_path"=>nil, "openshift_subdomain_name"=>nil, "openshift_sample_helloworld"=>false, "cloudforms_vcpu"=>nil, "cloudforms_ram"=>nil, "cloudforms_vm_disk_size"=>nil, "cloudforms_db_disk_size"=>nil, "created_at"=>"2016-07-21T14:45:36.559Z", "updated_at"=>"2016-07-21T14:45:36.559Z", "rhev_engine_host_id"=>nil, "organization_id"=>"1", "lifecycle_environment_id"=>nil, "discovered_host_id"=>nil, "foreman_task_id"=>nil, "openstack_deployment_id"=>"1"}, "api_version"=>"v21", "id"=>"1"}
2016-07-21 11:08:54 [app] [I] Processing by Fusor::Api::CustomerPortal::CustomerPortalProxiesController#get as JSON
2016-07-21 11:08:54 [app] [I]   Parameters: {"consumer"=>"e0429d10-0986-4aae-b008-ddb1f11d2cf6", "listall"=>"false", "uuid"=>"e0429d10-0986-4aae-b008-ddb1f11d2cf6"}
2016-07-21 11:08:54 [app] [I] Completed 200 OK in 28ms (Views: 5.0ms | ActiveRecord: 9.3ms)
2016-07-21 11:08:54 [app] [I] Started GET "/fusor/api/v21/subscriptions?deployment_id=1&source=added" for 10.10.116.69 at 2016-07-21 11:08:54 -0400
2016-07-21 11:08:54 [app] [I] Processing by Fusor::Api::V21::SubscriptionsController#index as JSON
2016-07-21 11:08:54 [app] [I]   Parameters: {"deployment_id"=>"1", "source"=>"added", "api_version"=>"v21"}
2016-07-21 11:08:54 [app] [I] Completed 200 OK in 5ms (Views: 0.7ms | ActiveRecord: 0.9ms)
2016-07-21 11:08:55 [app] [I] Completed 200 OK in 1452ms (Views: 6.8ms | ActiveRecord: 0.8ms)
2016-07-21 11:08:56 [app] [I] Completed 200 OK in 2050ms (Views: 5.3ms | ActiveRecord: 0.6ms)
2016-07-21 11:13:45 [foreman-tasks/dynflow] [I] start terminating throttle_limiter...

Comment 3 Thom Carlin 2016-07-21 16:02:26 UTC
Note: No user-visible feedback nor way to cancel this process.

Uploaded foreman-debug

Comment 5 Thom Carlin 2016-07-21 16:04:52 UTC
Refreshed page with same results.  foreman-tail did help me to find:
2016-07-21 12:03:27 [app] [I] Started GET "/javascripts/fusor_ui/application_no_tl.js" for 10.10.116.69 at 2016-07-21 12:03:27 -0400
2016-07-21 12:03:27 [app] [F] 
 | ActionController::RoutingError (No route matches [GET] "/javascripts/fusor_ui/application_no_tl.js"):
 |   actionpack (4.1.5) lib/action_dispatch/middleware/debug_exceptions.rb:21:in `call'
 |   actionpack (4.1.5) lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
 |   railties (4.1.5) lib/rails/rack/logger.rb:38:in `call_app'
 |   railties (4.1.5) lib/rails/rack/logger.rb:22:in `call'
 |   actionpack (4.1.5) lib/action_dispatch/middleware/request_id.rb:21:in `call'
 |   rack (1.5.2) lib/rack/methodoverride.rb:21:in `call'
 |   rack (1.5.2) lib/rack/runtime.rb:17:in `call'
 |   activesupport (4.1.5) lib/active_support/cache/strategy/local_cache_middleware.rb:26:in `call'
 |   actionpack (4.1.5) lib/action_dispatch/middleware/static.rb:64:in `call'
 |   actionpack (4.1.5) lib/action_dispatch/middleware/static.rb:64:in `call'
 |   actionpack (4.1.5) lib/action_dispatch/middleware/static.rb:64:in `call'
 |   rack (1.5.2) lib/rack/sendfile.rb:112:in `call'
 |   railties (4.1.5) lib/rails/engine.rb:514:in `call'
 |   railties (4.1.5) lib/rails/application.rb:144:in `call'
 |   railties (4.1.5) lib/rails/railtie.rb:194:in `public_send'
 |   railties (4.1.5) lib/rails/railtie.rb:194:in `method_missing'
 |   rack (1.5.2) lib/rack/builder.rb:138:in `call'
 |   rack (1.5.2) lib/rack/urlmap.rb:65:in `block in call'
 |   rack (1.5.2) lib/rack/urlmap.rb:50:in `each'
 |   rack (1.5.2) lib/rack/urlmap.rb:50:in `call'
 |   /usr/share/gems/gems/passenger-4.0.18/lib/phusion_passenger/rack/thread_handler_extension.rb:77:in `process_request'
 |   /usr/share/gems/gems/passenger-4.0.18/lib/phusion_passenger/request_handler/thread_handler.rb:140:in `accept_and_process_next_request'
 |   /usr/share/gems/gems/passenger-4.0.18/lib/phusion_passenger/request_handler/thread_handler.rb:108:in `main_loop'
 |   /usr/share/gems/gems/passenger-4.0.18/lib/phusion_passenger/request_handler.rb:441:in `block (3 levels) in start_threads'
 |   logging (1.8.2) lib/logging/diagnostic_context.rb:323:in `call'
 |   logging (1.8.2) lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context'
 | 
 |

Comment 6 Thom Carlin 2016-07-21 16:11:52 UTC
Was able to manually download the identity certificate and manifest successfully.

Comment 7 Thom Carlin 2016-07-21 16:15:02 UTC
Tried checking Org but ran into navigation bz

Comment 8 Thom Carlin 2016-07-21 16:19:22 UTC
Workaround:
1) Manually download manifest for SMA from Customer Portal
2) Manually upload manifest to Sat6
3) Restart deployment

Comment 9 John Matthews 2016-07-21 17:48:03 UTC
Thom is this something you are consistently seeing?

First question is if this is a QCI issue or is this a problem with the portal being down or impaired service.

Would help to know if this is a consistent issue you can reproduce.

Comment 10 Thom Carlin 2016-07-21 17:50:03 UTC
Per comment 5, it seems to be a combination of both

Comment 11 Thom Carlin 2016-07-21 18:36:51 UTC
Reproduced again with same manifest...trying a different "known good" manifest

Comment 12 Thom Carlin 2016-07-21 18:40:49 UTC
Same issue with "known good" manifest

Comment 13 Thom Carlin 2016-07-22 13:38:23 UTC
Tried clearing the cache (browser, cookies)

Comment 14 Erik Nelson 2016-07-22 18:38:57 UTC
It's possible a code path is taken that leaks the spinner, leaving it up infinitely. I have been unable to reproduce locally or on the machine in question.

I added the suspected error handler and ensured the spinner should come down + some diagnostics here: https://github.com/fusor/fusor/pull/1084

If this continues to be a problem, we should at least be able to narrow down the root cause with this patch.

NOTE: This may be somewhat difficult to reproduce.

Comment 15 John Matthews 2016-07-25 17:34:13 UTC
Moving this to post 1.0 ga.

If we are able to identify steps to reproduce we can re-examine for GA.

Comment 16 Thom Carlin 2016-07-26 14:03:58 UTC
Workaround with 07/25:
Click on "Review Subscriptions" link on left-hand-side.  This allows you to proceed

Comment 17 Erik Nelson 2016-08-30 17:07:31 UTC
We think this has been tracked down and patched with a workaround: https://github.com/fusor/fusor/pull/1197

See related bz: https://bugzilla.redhat.com/show_bug.cgi?id=1371616

Comment 18 Todd Sanders 2016-08-30 20:06:34 UTC

*** This bug has been marked as a duplicate of bug 1371616 ***


Note You need to log in before you can comment on or make changes to this bug.