Bug 1465325 - [3.5][Backport] Concurrent build registry push hangs - baremetal cluster with CNS Gluster registry storage
Summary: [3.5][Backport] Concurrent build registry push hangs - baremetal cluster with...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Image Registry
Version: 3.5.1
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
: 3.5.z
Assignee: Michal Minar
QA Contact: Hongkai Liu
URL:
Whiteboard: aos-scalability-35
Depends On: 1436841
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-06-27 08:38 UTC by Michal Minar
Modified: 2017-08-31 17:00 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: A bug in a regulator of concurrent filesystem access could cause a routine to hang. Consequence: When the registry configured with filesystem storage driver had been under heavy load, some could have hang forever. Fix: The regulator has been fixed. Result: The concurrent pushes no longer hang.
Clone Of: 1436841
Environment:
Last Closed: 2017-08-31 17:00:23 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:1828 0 normal SHIPPED_LIVE OpenShift Container Platform 3.5, 3.4, and 3.3 bug fix update 2017-08-31 20:59:56 UTC

Comment 1 Michal Minar 2017-06-27 08:47:11 UTC
PR with a back-port: https://github.com/openshift/ose/pull/798

Comment 2 Michal Minar 2017-07-21 06:22:44 UTC
Merged.

Comment 3 Mike Fiedler 2017-07-21 14:50:41 UTC
Should not go ON_QA until it is in a 3.5 puddle.  Moving back to MODIFIED

Comment 7 Hongkai Liu 2017-08-09 17:13:01 UTC
Verify with following setting:

REGISTRY_STORAGE_FILESYSTEM_MAXTHREADS=100
300 concurrent builds for several hours which consistently push to images to docker-reg.

Some build failed but no hanging occurred.

Comment 8 Mike Fiedler 2017-08-09 18:45:51 UTC
Verified on 3.5.5.31.12

Comment 9 Mike Fiedler 2017-08-09 18:47:08 UTC
Also, verified with the registry configured to use the filesystem storage driver.

Comment 11 errata-xmlrpc 2017-08-31 17:00:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1828


Note You need to log in before you can comment on or make changes to this bug.