Description of problem: The fact_values api called loaded a lot of un-needed data from the database and used a sub-optimal way of converting the data to a hash for display. Version-Release number of selected component (if applicable): Satellite 6.9.7 How reproducible: 100% Steps to Reproduce: Load all facts from hosts. https://sat-fqdn/apidoc/v2/fact_values Actual results: Sat6 instance with 300 hosts ~~~ 2021-11-25T17:51:57 [I|app|d7f0a4e0] Started GET "/api/v2/fact_values?per_page=9999999" for 10.208.172.104 at 2021-11-25 17:51:57 +0000 2021-11-25T17:51:57 [I|app|d7f0a4e0] Processing by Api::V2::FactValuesController#index as JSON 2021-11-25T17:51:57 [I|app|d7f0a4e0] Parameters: {"per_page"=>"9999999", "apiv"=>"v2", "fact_value"=>{}} 2021-11-25T17:51:57 [I|app|d7f0a4e0] Authorized user testing(testing ) 2021-11-25T17:52:35 [I|app|d7f0a4e0] Rendering api/v2/fact_values/index.json.rabl within api/v2/layouts/index_layout 2021-11-25T17:52:36 [I|app|d7f0a4e0] Rendered api/v2/fact_values/index.json.rabl within api/v2/layouts/index_layout (Duration: 206.9ms | Allocations: 237490) 2021-11-25T17:52:36 [I|app|d7f0a4e0] Completed 200 OK in 38748ms (Views: 219.0ms | ActiveRecord: 421.2ms | Allocations: 5964354) ~~~ Sat6 instance with 75 hosts same can be observed, althugh 6 seconds instead of 40 seconds ~~~ 2021-11-25T17:52:36 [I|app|99aecc01] Started GET "/api/v2/fact_values?per_page=9999999" for 10.208.172.104 at 2021-11-25 17:52:36 +0000 2021-11-25T17:52:36 [I|app|99aecc01] Processing by Api::V2::FactValuesController#index as JSON 2021-11-25T17:52:36 [I|app|99aecc01] Parameters: {"per_page"=>"9999999", "apiv"=>"v2", "fact_value"=>{}} 2021-11-25T17:52:36 [I|app|99aecc01] Authorized user testing(testing ) 2021-11-25T17:52:42 [I|app|99aecc01] Rendering api/v2/fact_values/index.json.rabl within api/v2/layouts/index_layout 2021-11-25T17:52:42 [I|app|99aecc01] Rendered api/v2/fact_values/index.json.rabl within api/v2/layouts/index_layout (Duration: 67.5ms | Allocations: 89770) 2021-11-25T17:52:42 [I|app|99aecc01] Completed 200 OK in 6062ms (Views: 76.6ms | ActiveRecord: 259.8ms | Allocations: 2226318) ~~~ Expected results: Better performance Additional info: All facts are requested in a single query for performance reasons.
Before the patch from https://github.com/theforeman/foreman/pull/9000.patch # curl -X GET -s -k -u admin:pass https://ktordeur-satellite.example.lan/api/v2/fact_values\?per_page\=99999999 | python -m json.tool ~~~ 2021-12-17T12:45:47 [I|app|85ed1fcd] Started GET "/api/v2/fact_values?per_page=99999999" for 10.40.194.212 at 2021-12-17 12:45:47 +0100 2021-12-17T12:45:47 [I|app|85ed1fcd] Processing by Api::V2::FactValuesController#index as JSON 2021-12-17T12:45:47 [I|app|85ed1fcd] Parameters: {"per_page"=>"99999999", "apiv"=>"v2"} 2021-12-17T12:45:47 [I|app|85ed1fcd] Authorized user admin(Admin User) 2021-12-17T12:45:48 [I|app|85ed1fcd] Rendering api/v2/fact_values/index.json.rabl within api/v2/layouts/index_layout 2021-12-17T12:45:48 [I|app|85ed1fcd] Rendered api/v2/fact_values/index.json.rabl within api/v2/layouts/index_layout (Duration: 23.2ms | Allocations: 12917) 2021-12-17T12:45:48 [I|app|85ed1fcd] Completed 200 OK in 650ms (Views: 42.1ms | ActiveRecord: 62.3ms | Allocations: 212369) ~~~ After the patch: ~~~ 2021-12-17T13:11:47 [I|app|82f50291] Started GET "/api/v2/fact_values?per_page=99999999" for 10.44.129.52 at 2021-12-17 13:11:47 +0100 2021-12-17T13:11:47 [I|app|82f50291] Processing by Api::V2::FactValuesController#index as JSON 2021-12-17T13:11:47 [I|app|82f50291] Parameters: {"per_page"=>"99999999", "apiv"=>"v2"} 2021-12-17T13:11:47 [I|app|82f50291] Authorized user admin(Admin User) 2021-12-17T13:11:47 [I|app|82f50291] Rendering api/v2/fact_values/index.json.rabl within api/v2/layouts/index_layout 2021-12-17T13:11:47 [I|app|82f50291] Rendered api/v2/fact_values/index.json.rabl within api/v2/layouts/index_layout (Duration: 11.0ms | Allocations: 12880) 2021-12-17T13:11:47 [I|app|82f50291] Completed 200 OK in 243ms (Views: 18.1ms | ActiveRecord: 32.5ms | Allocations: 138916) ~~~
Upstream bug assigned to tbrisker
Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/34160 has been resolved.
Thanks for the quick follow-up and preparing a code fix I applied the patch on 2 sat6 instances one smaller running 6.9.7 and larger one running 6.9.6 and can confirm it gives a big improvement: The 6.9.7 instance has 45000 facts goes from ~7sec -> ~1.5sec allocations ~6000000 -> 4000000 The 6.9.6 instance has 125000 facts goes from ~40sec -> ~5sec allocations 2200000 -> 1500000
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: Satellite 6.11 Release), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:5498