Bug 1029240 - Frequent errors from site test cases
Frequent errors from site test cases
Status: CLOSED CURRENTRELEASE
Product: OpenShift Online
Classification: Red Hat
Component: Website (Show other bugs)
2.x
Unspecified Unspecified
unspecified Severity medium
: ---
: ---
Assigned To: Fabiano Franz
libra bugs
: UpcomingRelease
: 1029678 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-11-11 19:05 EST by Dan McPherson
Modified: 2015-05-14 21:31 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-02-26 14:08:13 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Dan McPherson 2013-11-11 19:05:08 EST
From:

https://ci.dev.openshift.redhat.com/jenkins/job/test_pull_requests/5259/console

  1) Error:
test_cartridge_scale_parameters_can_be_changed(RestApiCartridgeTest):
RestApi::ServerUnavailable: Failed.  Response code = 503.  Response message = .
    /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-console-1.17.2/app/models/rest_api/base.rb:353:in `rescue in save_with_exception_handling'
    /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-console-1.17.2/app/models/rest_api/base.rb:335:in `save_with_exception_handling'
    /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-console-1.17.2/app/models/rest_api/base.rb:362:in `save_with_change_tracking'
    /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-console-1.17.2/test/integration/rest_api/cartridge_test.rb:47:in `block in <class:RestApiCartridgeTest>'

18 tests, 170 assertions, 0 failures, 1 errors, 0 skips
Coverage report generated for /opt/rh/ruby193/root/usr/share/gems/gems/rake-0.9.2.2/lib/rake/rake_test_loader.rb test/functional/cartridge_types_controller_test.rb /opt/rh/ruby193/root/usr/share/gems/gems/ci_reporter-1.7.2/lib/ci/reporter/rake/minitest_loader.rb, /opt/rh/ruby193/root/usr/share/gems/gems/rake-0.9.2.2/lib/rake/rake_test_loader.rb test/functional/cartridges_controller_test.rb /opt/rh/ruby193/root/usr/share/gems/gems/ci_reporter-1.7.2/lib/ci/reporter/rake/minitest_loader.rb, /opt/rh/ruby193/root/usr/share/gems/gems/rake-0.9.2.2/lib/rake/rake_test_loader.rb test/integration/rest_api/membership_test.rb test/integration/rest_api/cartridge_test.rb test/integration/rest_api/cartridge_type_test.rb /opt/rh/ruby193/root/usr/share/gems/gems/ci_reporter-1.7.2/lib/ci/reporter/rake/minitest_loader.rb, /opt/rh/ruby193/root/usr/share/gems/gems/rake-0.9.2.2/lib/rake/rake_test_loader.rb test/integration/status_app_test.rb test/integration/rescue_from_test.rb test/integration/settings_flows_test.rb test/integration/singular_resources_test.rb test/integration/quickstarts_test.rb test/integration/assets_test.rb test/integration/static_pages_test.rb test/unit/helpers/home_helper_test.rb test/unit/helpers/model_helper_test.rb test/unit/helpers/country_helper_test.rb test/unit/helpers/application_helper_test.rb test/unit/helpers/account_helper_test.rb test/unit/helpers/express_domain_helper_test.rb test/unit/helpers/product_helper_test.rb test/unit/community_api_test.rb test/unit/async_aware_test.rb test/unit/download_test.rb test/unit/configuration_test.rb test/unit/aria_test.rb test/unit/log_subscriber_test.rb test/unit/tweet_test.rb test/unit/rest_api_test.rb test/unit/cost_aware_test.rb test/unit/streamline_test.rb test/unit/capability_aware_test.rb test/unit/bootstrap_form_builder_test.rb test/unit/overrides_test.rb test/unit/active_model_compliance_test.rb test/unit/web_user_test.rb test/unit/cloud_access_test.rb test/unit/faq_item_test.rb test/unit/filter_hash_test.rb /opt/rh/ruby193/root/usr/share/gems/gems/ci_reporter-1.7.2/lib/ci/reporter/rake/minitest_loader.rb to /root/openshift-test/site/test/coverage/. 5191 / 7562 LOC (68.65%) covered.
rake aborted!
Command failed with status (1): [/opt/rh/ruby193/root/usr/bin/ruby -I"lib:t...]



How reproducible:

60%


And 99% of the time this works fine on retry.  Decent chance this is a problem below the site.  If so please provide the context of the issue.
Comment 1 Andy Goldstein 2013-11-12 12:48:00 EST
app uuid 5280fda8873c793f3c000094

At 10:55:58, it looks like a scale up to 3 was started (PUT "/broker/rest/application/5280fda8873c793f3c000094/cartridge/ruby-1.8.json) (pid 5632).

At 10:57:19, update-cluster was invoked via MCollective (pid 5632).

At 10:57:20, another request came in (also PUT "/broker/rest/application/5280fda8873c793f3c000094/cartridge/ruby-1.8.json) (pid 28292) setting the scale min/max to 2

At 10:58:10, the 2nd scale request failed - Unable to perform action. Another operation is already running. (pid:28292)

At 10:58:37, update-cluster for pid 5632 finished
Comment 2 Jordan Liggitt 2014-01-16 15:26:38 EST
*** Bug 1029678 has been marked as a duplicate of this bug. ***
Comment 3 Fabiano Franz 2014-01-23 11:41:20 EST
Fixed in https://github.com/openshift/li/pull/2300
Comment 4 openshift-github-bot 2014-01-23 13:27:55 EST
Commit pushed to master at https://github.com/openshift/li

https://github.com/openshift/li/commit/6f7208d3448261bf741fcc5a7fb7833c74ed3b0f
Bug 1029240 - moved REST API cartridge test to avoid race conditions when scaling with_scalable_app in different suites
Comment 5 Yujie Zhang 2014-01-24 00:24:28 EST
Tested on devenv_4269 and ran the cartridge_test.rb, pass without error, result is as following:

[root@ip-10-138-38-81 rest_api]# ruby cartridge_test.rb 

*** Mocha deprecation warning: Change `require 'mocha'` to `require 'mocha/setup'`.

Run options: --seed 14072

# Running tests:

..

Finished tests in 147.932346s, 0.0135 tests/s, 0.1082 assertions/s.

2 tests, 16 assertions, 0 failures, 0 errors, 0 skips
Coverage report generated for cartridge_test.rb to /var/www/openshift/site/test/integration/rest_api/test/coverage/. 3346 / 7651 LOC (43.73%) covered.

So verify this bug, thanks.

Note You need to log in before you can comment on or make changes to this bug.