Bing Li, could you please also upload registry logs from the time of the failed push? Anyway, this doesn't look like a registry issue. It's rather an issue with remote s3 bucket.
Created attachment 1142091 [details] Log of Docker registry, filtered by project name: binglitest0331
Created attachment 1143385 [details] s3 push
I cannot reproduce this either locally or using s3. (See attachment above and pastebin here: http://pastebin.com/y3Lg3v7P). Not sure if it will be of any help but can you post your s3 configuration?
Is this happening consistently in your environment and if so can Michail have access to it?
When I tried to push images from my computer in beijing, it would fail frequently, like this: http://pastebin.com/M4M8z81Z. Until after I pushed 5 times, then it succeeded. But when I tried to push the images from an AWS instance, it always succeeded. This issue never happened again. So maybe it's not a registry issue.
I'm inclined to say this is similar to bug 1321968, which might be periodic AWS problems. I've went through the registry logs and I can't see any anomalies there, all the s3-related operations are quite instantaneous. I'd propose lowering prio of this bug but keep it open until it appears again. There seems to be some problems and we need to track them down, definitely. @Bing Li: next time when attaching docker registry logs, please attach full log from the time the timeouts happen +/- 5 mins. It might be that grepping per project might cut some important messages we're missing here.
With the new docker registry [1] comes different s3 driver, specifically it's using AWS driver instead of external one. According to tests done by Michal the performance should be greatly improved. I'm moving this bug to QA, similarly to bug 1314381, comment 24. [1] https://github.com/openshift/origin/pull/8938
confirmed with fork_ami_openshift3_miminar_295, and can not reproduce this issue