Bug 865841 - When several clients request updates some clients can't get packages
When several clients request updates some clients can't get packages
Status: CLOSED ERRATA
Product: Subscription Asset Manager
Classification: Red Hat
Component: thumbslug (Show other bugs)
1.3
Unspecified Unspecified
unspecified Severity unspecified
: rc
: 1.3
Assigned To: Katello Bug Bin
Tazim Kolhar
:
Depends On:
Blocks: sam13-tracker
  Show dependency treegraph
 
Reported: 2012-10-12 11:37 EDT by Og Maciel
Modified: 2016-04-25 20:54 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-10-01 06:50:01 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Og Maciel 2012-10-12 11:37:11 EDT
Description of problem:

So I have a SAM 1.2 system with 8 real clients properly registered and subscribed, running alongside several other VMS... it can get a bit slow for sure. Anyhow, I tried to install httpd in all of them inside a loop:

for i in 2 3 4 5 6 7 8 9; do ssh root@qeclient0$i.usersys.redhat.com 'yum install -y httpd'; done

Some of the clients got httpd installed, some failed with:

Downloading Packages:
https://qetello03.usersys.redhat.com:8088/content/dist/rhel/server/6/6Server/x86_64/os/Packages/apr-1.3.9-5.el6_2.x86_64.rpm: [Errno 14] problem making ssl connection
Trying other mirror.


Error Downloading Packages:
  apr-1.3.9-5.el6_2.x86_64: failure: Packages/apr-1.3.9-5.el6_2.x86_64.rpm from rhel-6-server-rpms: [Errno 256] No more mirrors to try.

Version-Release number of selected component (if applicable):

* candlepin-0.7.13-1.el6_3.noarch
* candlepin-tomcat6-0.7.13-1.el6_3.noarch
* katello-candlepin-cert-key-pair-1.0-1.noarch
* katello-certs-tools-1.1.8-1h.el6_3.noarch
* katello-cli-1.1.11-1h.el6_3.noarch
* katello-cli-common-1.1.11-1h.el6_3.noarch
* katello-common-1.1.15-2h.el6_3.noarch
* katello-configure-1.1.11-1h.el6_3.noarch
* katello-glue-candlepin-1.1.15-2h.el6_3.noarch
* katello-headpin-1.1.15-2h.el6_3.noarch
* katello-headpin-all-1.1.15-2h.el6_3.noarch
* katello-selinux-1.1.2-1h.el6_3.noarch
* thumbslug-0.0.24-1.el6_2.noarch
* thumbslug-selinux-0.0.24-1.el6_2.noarch

How reproducible:


Steps to Reproduce:
1. Registered and subscribed 8 clients (all RHEL 62. x86_64) to my organization
2. Upload valid manifest
3. get all 8 clients to install something, such as firefox or httpd
  
Actual results:


Expected results:


Additional info:

fwiw this is a SAM 1.2 VM with 4GB RAM and sharing space with other VMS.

I also noticed thumbslug:

Oct 11 16:21:47 [New I/O server worker #1-6] ERROR org.candlepin.thumbslug.HttpRequestHandler - Exception caught!
java.io.IOException: Connection reset by peer
»···at sun.nio.ch.FileDispatcher.read0(Native Method)
»···at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
»···at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:251)
»···at sun.nio.ch.IOUtil.read(IOUtil.java:218)
»···at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:254)
»···at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:323)
»···at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
»···at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:202)
»···at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
»···at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
»···at java.lang.Thread.run(Thread.java:679)
Oct 11 17:00:58 [New I/O server worker #1-2] ERROR org.candlepin.thumbslug.HttpRequestHandler - Exception caught!
java.io.IOException: Broken pipe
»···at sun.nio.ch.FileDispatcher.write0(Native Method)
»···at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
»···at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:122)
»···at sun.nio.ch.IOUtil.write(IOUtil.java:78)
»···at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:352)
»···at org.jboss.netty.channel.socket.nio.SocketSendBufferPool$PooledSendBuffer.transferTo(SocketSendBufferPool.java:240)
»···at org.jboss.netty.channel.socket.nio.NioWorker.write0(NioWorker.java:472)
»···at org.jboss.netty.channel.socket.nio.NioWorker.writeFromSelectorLoop(NioWorker.java:402)
»···at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:288)
»···at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:202)
»···at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
»···at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
»···at java.lang.Thread.run(Thread.java:679)

catalina.out shows:

Oct 12 10:52:15 [http-8443-8] WARN  org.candlepin.jackson.JsonBeanPropertyFilter - Unable to serialize property 'autoheal without getter: getAutoheal
Oct 12 10:52:16 [http-8443-9] WARN  org.candlepin.jackson.JsonBeanPropertyFilter - Unable to serialize property 'canActivate without getter: getCanActivate
Oct 12 10:52:16 [http-8443-9] WARN  org.candlepin.jackson.JsonBeanPropertyFilter - Unable to serialize property 'autoheal without getter: getAutoheal
Oct 12 10:52:16 [http-8443-6] WARN  org.candlepin.jackson.JsonBeanPropertyFilter - Unable to serialize property 'canActivate without getter: getCanActivate
Oct 12 10:52:16 [http-8443-6] WARN  org.candlepin.jackson.JsonBeanPropertyFilter - Unable to serialize property 'autoheal without getter: getAutoheal
Oct 12 10:52:16 [http-8443-2] WARN  org.candlepin.jackson.JsonBeanPropertyFilter - Unable to serialize property 'canActivate without getter: getCanActivate
Oct 12 10:52:16 [http-8443-2] WARN  org.candlepin.jackson.JsonBeanPropertyFilter - Unable to serialize property 'autoheal without getter: getAutoheal
Comment 1 Og Maciel 2012-10-25 10:04:11 EDT
I tried this same experiment with a "beefy" VM that had 3 times as much RAM and using latest CFSE. After registering and subscribing 10 systems, I had no issue consuming content when making 10 simultaneous requests. Will have to perform the same experiment using SAM with more RAM (or less VMs competing for resources).
Comment 2 Og Maciel 2012-10-31 14:28:14 EDT
Re-tested SAM 1.2 with 10 simultaneous requests, all passed w/o a glitch
Comment 3 James Bowes 2012-11-07 12:24:12 EST
I've done some more work to help with concurrency in thumbslug, though ultimately we're limited by your system's cpu's ability do process ssl connections, and how many concurrent connections from the same system the CDN will accept.

Anyhoo, calling this one FIXED in thumbslug master, 01f85744de1ef3e12373b065
should be in thumbslug 0.0.27
Comment 4 Bryan Kearney 2013-06-07 14:20:17 EDT
Moving all POST bugs to ON_QA since we have delivered a puddle with the bugs.
Comment 5 Tazim Kolhar 2013-08-13 12:20:49 EDT
VERIFIED :

# rpm -qa | grep katello
katello-selinux-1.4.4-2.el6sat.noarch
katello-candlepin-cert-key-pair-1.0-1.noarch
katello-certs-tools-1.4.2-2.el6sat.noarch
katello-cli-common-1.4.3-5.el6sat.noarch
katello-cli-1.4.3-5.el6sat.noarch
katello-common-1.4.3-6.el6sam_splice.noarch
katello-configure-1.4.4-2.el6sat.noarch
katello-glue-elasticsearch-1.4.3-6.el6sam_splice.noarch
katello-headpin-all-1.4.3-6.el6sam_splice.noarch
katello-glue-candlepin-1.4.3-6.el6sam_splice.noarch
signo-katello-0.0.10-2.el6sat.noarch
katello-headpin-1.4.3-6.el6sam_splice.noarch


1.)Upload valid manifest on Sam server

2.)Registered and subscribed 8 clients (all RHEL 62. x86_64) to my organization
   
    i. rpm -Uvh http://SAM_SERVER/pub/candlepin-cert-consumer-latest.noarch.rpm
Retrieving http://SAM_SERVER/pub/candlepin-cert-consumer-latest.noarch.rpm
Preparing...                ########################################### [100%]
   1:candlepin-cert-consumer########################################### [100%]


   ii.# subscription-manager register --org=ACME_Corporation --autosubscribe


3.) got all 8 clients to install firefox
    
   
4.) run firefox &  
    # firefox &
      [1] 15297
Comment 7 errata-xmlrpc 2013-10-01 06:50:01 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2013-1390.html

Note You need to log in before you can comment on or make changes to this bug.