Bug 2121288

Summary: Still getting API request timeout when indexing contents.
Product: Red Hat Satellite Reporter: Hao Chang Yu <hyu>
Component: PackagingAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED ERRATA QA Contact: Lukas Pramuk <lpramuk>
Severity: high Docs Contact:
Priority: high    
Version: 6.11.2CC: ahumbe, ankulkar, dalley, dkliban, ggainey, jsenkyri, Ken.Fowler, ngalvin, osousa, pcreech, pmoravec, rchan, risantam, sadas, saydas, torkil, wclark, wpinheir
Target Milestone: 6.13.0Keywords: PrioBumpGSS, Regression, Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: pulp_rpm_client-3.17.10-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2131783 (view as bug list) Environment:
Last Closed: 2023-05-03 13:21:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Hotfix RPM for Satellite 6.11.3 on RHEL7
none
Hotfix RPM for Satellite 6.11.3 on RHEL8 none

Description Hao Chang Yu 2022-08-25 05:05:14 UTC
Description of problem:
Bug 2033853 didn't fix the Pulp client API timeout error due to missing one code fix in "/usr/share/gems/gems/pulp_rpm_client-3.17.4/lib/pulp_rpm_client/api_client.rb" file.


Steps to reproduce:
1) Make the API to sleep for more than 60 seconds:
~~~
class ContentRepositoryVersionFilter(RepoVersionHrefFilter):
    """
    Filter used to get the content of this type found in a repository version.
    """

    def filter(self, qs, value):
        """
        Args:
            qs (django.db.models.query.QuerySet): The Content Queryset
            value (string): The RepositoryVersion href to filter by

        Returns:
            Queryset of the content contained within the specified repository version
        """
        if value is None:
            # user didn't supply a value
            return qs


        import time   <============== Add these 2 lines
        time.sleep(70) <=============

        repo_version = self.get_repository_version(value)
        return qs.filter(pk__in=repo_version.content)
~~~

2) Restart Pulpcore to take effect:
~~~
systemctl restart pulpcore*
~~~


3) Call API to get packages in foreman-rake console:
~~~
foreman-rake console
irb(main):039:0> rpm_api = PulpRpmClient::ContentPackagesApi.new(Katello::Pulp3::Api::Yum.new(SmartProxy.pulp_primary!).api_client)
irb(main):040:0> rpm_api.list({ "offset" => 0, repository_version: Katello::Repository.first.version_href })
Traceback (most recent call last):
        3: from lib/tasks/console.rake:5:in `block in <top (required)>'
        2: from (irb):37
        1: from (irb):38:in `rescue in irb_binding'
PulpRpmClient::ApiError (Error message: the server returns an error)
~~~


Additional info:
"/usr/share/gems/gems/pulp_rpm_client-3.17.4/lib/pulp_rpm_client/api_client.rb"
~~~
    def build_request(http_method, path, request, opts = {})
      url = build_request_url(path)
      http_method = http_method.to_sym.downcase

      header_params = @default_headers.merge(opts[:header_params] || {})
      query_params = opts[:query_params] || {}
      form_params = opts[:form_params] || {}

      update_params_for_auth! header_params, query_params, opts[:auth_names]

      req_opts = {
        :method => http_method,
        :headers => header_params,
        :params => query_params,
        :params_encoding => @config.params_encoding,
        :timeout => @config.timeout,
        :verbose => @config.debugging
      }

      if [:post, :patch, :put, :delete].include?(http_method)
        req_body = build_request_body(header_params, form_params, opts[:body])
        req_opts.update :body => req_body
        if @config.debugging
          @config.logger.debug "HTTP request body param ~BEGIN~\n#{req_body}\n~END~\n"
        end
      end
      request.headers = header_params
      request.body = req_body
      request.options = OpenStruct.new(req_opts)  <============ This line is missing in Satellite 6.11 so the timeout is not passed to the request.
      request.url url
      request.params = query_params
~~~

This issue is fixed in Satellite 6.10 because it included the above line in "build_request" method.

Comment 5 Daniel Alley 2022-09-13 12:11:13 UTC
Actually based on the latest comments (which happened after our dicussion) it's plausible this will be fixed by https://bugzilla.redhat.com/show_bug.cgi?id=2122344 in 6.12

Comment 6 Hao Chang Yu 2022-09-14 04:51:50 UTC
The missing line mentioned in comment 0 is still valid to prevent the 60 seconds client side timeout for slow systems so I think this bug is to fix the timeout setting.

Comment 7 Daniel Alley 2022-09-14 12:30:41 UTC
Ah, then this is a bindings issue.  I will bring it up with Dennis.

Comment 8 Dennis Kliban 2022-09-14 15:57:13 UTC
Satellite 6.11.2 ships pulp_rpm_client 3.17.4. This package does not contain the timeout fix. There is currently no pulp_rpm_client 3.17.z that contains this fix. We need to release pulp_rpm and pulp_rpm_client 3.17.13 in order to make this fix available for pulp_rpm 3.17.

Comment 9 Dennis Kliban 2022-09-14 18:13:33 UTC
My previous comment is incorrect. The fix should be present starting with pulp_rpm_client 3.17.5. This package is available on PyPI and just needs to be released as an RPM.

Comment 10 Daniel Alley 2022-09-23 15:53:39 UTC
Transitioning to a packaging issue per Dennis' comment

Comment 11 Daniel Alley 2022-09-29 15:55:34 UTC
Marking the fixed-in-version, however, in the future we should probably just have the API client bindings package versions match with the version of components which we are using.

Comment 12 wclark 2022-10-14 19:53:37 UTC
Created attachment 1918150 [details]
Hotfix RPM for Satellite 6.11.3 on RHEL7

INSTALL INSTRUCTIONS:

1. Take a complete backup or snapshot of Satellite 6.11.3 server

2. Download the Hotfix RPM for Satellite 6.11.3 on RHEL7 attached to this BZ and copy it to Satellite server

3. # yum install ./tfm-rubygem-pulp_rpm_client-3.17.10-0.HOTFIXRHBZ2121288.el7sat.noarch.rpm --disableplugin=foreman-protector

4. # satellite-maintain service restart

Comment 13 wclark 2022-10-14 19:55:18 UTC
Created attachment 1918151 [details]
Hotfix RPM for Satellite 6.11.3 on RHEL8

INSTALL INSTRUCTIONS:

1. Take a complete backup or snapshot of Satellite 6.11.3 server

2. Download the Hotfix RPM for Satellite 6.11.3 on RHEL8 attached to this BZ and copy it to Satellite server

3. # dnf install ./rubygem-pulp_rpm_client-3.17.10-0.HOTFIXRHBZ2121288.el8sat.noarch.rpm --disableplugin=foreman-protector

4. # satellite-maintain service restart

Comment 15 Lukas Pramuk 2023-03-22 11:29:03 UTC
VERIFIED.

@Satellite 6.13.0 Snap15
rubygem-pulp_rpm_client-3.18.7-1.el8sat.noarch

by the following steps:

# rpm -q rubygem-pulp_rpm_client
rubygem-pulp_rpm_client-3.18.7-1.el8sat.noarch

>>> new rubygem fixing the issue is present

# grep -C2 'request\.options' /usr/share/gems/gems/pulp_rpm_client-*/lib/pulp_rpm_client/api_client.rb
      request.headers = header_params
      request.body = req_body
      request.options = OpenStruct.new(req_opts)     <<<<<
      request.url url
      request.params = query_params

>>> the line is not missing so the timeout should be passed to the request.

Comment 20 errata-xmlrpc 2023-05-03 13:21:46 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.13 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:2097