Bug 1275647

Summary: Proxy templates API breaks on concurrent requests
Product: Red Hat Satellite Reporter: Bryan Kearney <bkearney>
Component: Foreman ProxyAssignee: Dmitri Dolguikh <ddolguik>
Status: CLOSED ERRATA QA Contact: Sachin Ghai <sghai>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.0.4CC: bbuckingham, chpeters, cwelton, ddolguik, jmontleo, juwu, kboumedh, lzap, mmccune, sghai
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
URL: http://projects.theforeman.org/issues/12319
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
A race condition existed on the capsule when generating templates. The proxy was improved to support concurrent releases.
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-01-21 07:42:34 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
test_result with 8443 port none

Description Bryan Kearney 2015-10-27 12:25:20 UTC
When trying to render templates concurrently, multiple (different) failures occurs.

Reproducer:

<pre>
#!/bin/bash

proxy_url=$1
token=$2
num_requests=$3

path="${proxy_url}/unattended/iPXE?token=${token}"

for (( i=0; i<$num_requests; i++ ))
do
curl -s $path &
done
</pre>

<pre>
# ./proxy_error_test.sh http://localhost:8448 eb9e1d9c-0bd1-4361-a3c2-7e0961a9b5b6 15

Failed to retrieve iPXE template for {"token"=>"eb9e1d9c-0bd1-4361-a3c2-7e0961a9b5b6", "splat"=>[], "captures"=>["iPXE"], "kind"=>"iPXE"}: stream closedFailed to retrieve iPXE template for {"token"=>"eb9e1d9c-0bd1-4361-a3c2-7e0961a9b5b6", "splat"=>[], "captures"=>["iPXE"], "kind"=>"iPXE"}: stream closedFailed to retrieve iPXE template for {"token"=>"eb9e1d9c-0bd1-4361-a3c2-7e0961a9b5b6", "splat"=>[], "captures"=>["iPXE"], "kind"=>"iPXE"}: undefined method `closed?' for nil:NilClassFailed to retrieve iPXE template for {"token"=>"eb9e1d9c-0bd1-4361-a3c2-7e0961a9b5b6", "splat"=>[], "captures"=>["iPXE"], "kind"=>"iPXE"}: undefined method `closed?' for nil:NilClassFailed to retrieve iPXE template for {"token"=>"eb9e1d9c-0bd1-4361-a3c2-7e0961a9b5b6", "splat"=>[], "captures"=>["iPXE"], "kind"=>"iPXE"}: undefined method `closed?' for nil:NilClassFailed to retrieve iPXE template for {"token"=>"eb9e1d9c-0bd1-4361-a3c2-7e0961a9b5b6", "splat"=>[], "captures"=>["iPXE"], "kind"=>"iPXE"}: undefined method `closed?' for nil:NilClassFailed to retrieve iPXE template for {"token"=>"eb9e1d9c-0bd1-4361-a3c2-7e0961a9b5b6", "splat"=>[], "captures"=>["iPXE"], "kind"=>"iPXE"}: undefined method `closed?' for nil:NilClass
</pre>

Comment 1 Bryan Kearney 2015-10-27 12:25:21 UTC
Created from redmine issue http://projects.theforeman.org/issues/12319

Comment 3 Lukas Zapletal 2015-11-02 09:17:41 UTC
I think this should make it in 6.1 z-stream. Upstream merged. Low effort.

Comment 4 Bryan Kearney 2015-11-02 11:02:57 UTC
Upstream bug assigned to ddolguik

Comment 5 Bryan Kearney 2015-11-02 11:03:00 UTC
Moving to POST since upstream bug http://projects.theforeman.org/issues/12319 has been closed
-------------
Anonymous
Applied in changeset commit:26aba8520d3d32ca27b9708cb98169bc31d43d7b.

Comment 11 Sachin Ghai 2016-01-18 07:21:29 UTC
I was trying to verify this bz with given script and steps in comment 7 and 8. However, looks like port 8000 is not valid. 

hwoever, with port 8443, I'm getting following output.  

[root@cloud-qe-6 ~]# ./racetest.sh http://localhost:8443 b9d50b5c-23a3-4a66-b4d2-9f18cafc7db9 5
[root@cloud-qe-6 ~]# 





Is this expected result, please confirm ? thanks

Comment 12 Sachin Ghai 2016-01-18 07:23:06 UTC
Created attachment 1115759 [details]
test_result with 8443 port

Comment 13 Sachin Ghai 2016-01-18 07:23:39 UTC
Please take a look at attachment.

Comment 14 Lukas Zapletal 2016-01-18 08:56:38 UTC
Oh port is the smart proxy port, which is usually 9090, 8448 or 8443 according to your configuration. You must see request logs in proxy.log.

Comment 15 Sachin Ghai 2016-01-18 10:07:32 UTC
Thank you Lukas.

Verified with in 6.1.6 compose #5

On concurrent requests, I don't see error reported in original bug request. I tried with 1500 requests concurrently.

[root@cloud-qe-6 ~]# ./racetest.sh http://localhost:9090 b9e1d9c-0bd1-4361-a3c2-7e0961a9b5b6 1500
[root@cloud-qe-6 ~]# ./racetest.sh https://localhost:9090 b9e1d9c-0bd1-4361-a3c2-7e0961a9b5b6 1500
Failed to retrieve iPXE template for {"token"=>"b9e1d9c-0bd1-4361-a3c2-7e0961a9b5b6", "splat"=>[], "captures"=>["iPXE"], "kind"=>"iPXE"}: Error retrieving iPXE for {"token"=>"b9e1d9c-0bd1-4361-a3c2-7e0961a9b5b6", :url=>"http://cloud-qe-6.idmqe.lab.eng.bos.redhat.com:8000"} from cloud-qe-6.idmqe.lab.eng.bos.redhat.com: Net::HTTPMethodNotAllowed
:::::
:::::
Failed to retrieve iPXE template for {"token"=>"b9e1d9c-0bd1-4361-a3c2-7e0961a9b5b6", "splat"=>[], "captures"=>["iPXE"], "kind"=>"iPXE"}: Error retrieving iPXE for {"token"=>"b9e1d9c-0bd1-4361-a3c2-7e0961a9b5b6", :url=>"http://cloud-qe-6.idmqe.lab.eng.bos.redhat.com:8000"} from cloud-qe-6.idmqe.lab.eng.bos.redhat.com: Net::HTTPMethodNotAllowed

Comment 19 errata-xmlrpc 2016-01-21 07:42:34 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:0052

Comment 20 Pavel Moravec 2016-06-20 10:10:57 UTC
*** Bug 1343667 has been marked as a duplicate of this bug. ***