Description of problem: GET requests like /candlepin/consumers/7ff6bec6-2e52-4d99-9d40-7223d27f6a5e/guests takes above 2minutes at a customer, flooding passenger queue of Satellite. Running the queries on an idle candlepin, it takes 1m13s and postgres is full of long-running statements like: 2018-03-12 11:15:15 EDT LOG: duration: 369.689 ms execute <unnamed>: select cp_consumer.id from cp_consumer inner join cp_consumer_facts on cp_consumer.id = cp_consumer_facts.cp_consumer_id where cp_consumer_facts.mapkey = 'virt.uuid' and lower(cp_consumer_facts.element) in ($1, $2) and cp_consumer.owner_id = $3 order by cp_consumer.updated desc 2018-03-12 11:15:15 EDT DETAIL: parameters: $1 = '42023592-dceb-9e50-3642-479ef1597a63', $2 = '92350242-ebdc-509e-3642-479ef1597a63', $3 = '8abba470582f8d8a01582f8e14010001' (cp_consumer having 8k records, cp_consumer_facts having 1M records) Version-Release number of selected component (if applicable): Sat 6.2.12 candlepin-0.9.54.23-1.el7.noarch How reproducible: 100% Steps to Reproduce: 1. (no idea of generic reproducer, just on the customer data we have) 2. time curl -u admin:admin -ks https://localhost:8443/candlepin/consumers/145b68c1-8d5a-480b-9e06-d1bd042188da/guests > out.txt Actual results: takes over 1 minute Expected results: takes several seconds ideally Additional info: candlepin postgres dump will be provided
workaround is in adding an index, per KCS 3379731
Different reproducer: run more requests in parallel such that some such requests are against the same type of URI ( consumer/*/guests ). Running one such request on idle candlepin, it took 8-10s to complete. Running it concurrently with some other / random requests (max 10 concurrently run), it took up to even 160s to complete it. So another use case is to run the guest request with some other requests in parallel.
With no way to reproduce, I am marking this bug verified under no-break conditions.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2019:1222