Bug 2049505

Summary: openSCAP scan failed with error Upload failed: Net::ReadTimeout
Product: Red Hat Satellite Reporter: isinghal
Component: SCAP PluginAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED DUPLICATE QA Contact: Jameer Pathan <jpathan>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.10.2CC: ikaur, jhutar, mhulan, pmendezh, rlavi, visawant
Target Milestone: UnspecifiedKeywords: Performance, Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-09-04 12:54:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
error message from UI
none
ps
none
top none

Description isinghal 2022-02-02 11:16:06 UTC
Description of problem: Uploading openSCAP scan reports to satellite fails on 80% hosts when running on multiple hosts


Version-Release number of selected component (if applicable): Sat 6.10


How reproducible: Always


Steps to Reproduce:
1. Select 100 hosts on satellite and schedule openSCAP scan job
2. Monitor the job and see that most host fail


Actual results: While the scan runs fine, most hosts fail to upload result back to satellite with the error 'Upload failed: Net::ReadTimeout'. 


Expected results: All hosts should be scanned and report uploaded back to satellite.


Additional info: Looking the resources used during scan, we feel this may be an issue of excess memory usage by ruby due to reports being unzipped on satellite (we tested this on a 43G machine).
foreman-debug: http://perf54.perf.lab.eng.bos.redhat.com/pub/foreman-debug-vdtsw.tar.xz

Comment 1 isinghal 2022-02-25 08:05:07 UTC
Created attachment 1863276 [details]
error message from UI

Comment 2 isinghal 2022-02-25 08:07:34 UTC
Created attachment 1863277 [details]
ps

Comment 3 isinghal 2022-02-25 08:08:31 UTC
Created attachment 1863278 [details]
top

Comment 4 Marek Hulan 2022-02-25 09:35:30 UTC
Try to increase the timeout in /etc/foreman-proxy/settings.d/openscap.yml, if that helps it should be reconfigured through satellite-installer. When you schedule a scan on many machines like this, it's good to use the scheduling capability of the remote execution job. In the job form, click "Display advanced fields" and set either the Concurrency level or Time span. That should help keeping the load under control.

Comment 7 Brad Buckingham 2023-09-01 21:10:52 UTC
Upon review of our valid but aging backlog the Satellite Team has concluded that this Bugzilla does not meet the criteria for a resolution in the near term, and are planning to close in a month. This message may be a repeat of a previous update and the bug is again being considered to be closed. If you have any concerns about this, please contact your Red Hat Account team.  Thank you.

Comment 8 Ron Lavi 2023-09-04 12:54:56 UTC
Marking it as a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1893672
and the reason might be in https://bugzilla.redhat.com/show_bug.cgi?id=2237152 (not yet confirmed)

*** This bug has been marked as a duplicate of bug 1893672 ***