Bug 1647125 - Calls to AWS provider go over AWS 200 item filter limit
Summary: Calls to AWS provider go over AWS 200 item filter limit
Keywords:
Status: CLOSED DUPLICATE of bug 1612086
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Providers
Version: 5.9.3
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: GA
: 5.9.6
Assignee: Bronagh Sorota
QA Contact: Matouš Mojžíš
URL:
Whiteboard: ec2
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-11-06 17:37 UTC by Robb Manes
Modified: 2021-12-10 18:11 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-11-19 18:42:57 UTC
Category: ---
Cloudforms Team: AWS
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Robb Manes 2018-11-06 17:37:34 UTC
Description of problem:
If attempting to get images from an AWS provider, and a call is made to a method that does filtering (as an example, `referenced_images`), AWS will return an error expressing it does not allow over 200 filtered items.

[----] E, [2018-10-31T04:36:12.585801 #24179:e33118] ERROR -- : MIQ(ManageIQ::Providers::Amazon::CloudManager::Refresher#refresh) EMS: [AWS-Product-Dev], id: [1000000000003] Refresh failed
[----] E, [2018-10-31T04:36:12.586062 #24179:e33118] ERROR -- : [Aws::EC2::Errors::FilterLimitExceeded]: The maximum number of filter values specified on a single call is 200  Method:[block in method_missing]
[----] E, [2018-10-31T04:36:12.586166 #24179:e33118] ERROR -- : /opt/rh/cfme-gemset/gems/aws-sdk-core-2.9.44/lib/seahorse/client/plugins/raise_response_errors.rb:15:in `call'
/opt/rh/cfme-gemset/gems/aws-sdk-core-2.9.44/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:20:in `call'
/opt/rh/cfme-gemset/gems/aws-sdk-core-2.9.44/lib/aws-sdk-core/plugins/idempotency_token.rb:18:in `call'
/opt/rh/cfme-gemset/gems/aws-sdk-core-2.9.44/lib/aws-sdk-core/plugins/param_converter.rb:20:in `call'
/opt/rh/cfme-gemset/gems/aws-sdk-core-2.9.44/lib/aws-sdk-core/plugins/response_paging.rb:26:in `call'
/opt/rh/cfme-gemset/gems/aws-sdk-core-2.9.44/lib/seahorse/client/plugins/response_target.rb:21:in `call'
/opt/rh/cfme-gemset/gems/aws-sdk-core-2.9.44/lib/seahorse/client/request.rb:70:in `send_request'
/opt/rh/cfme-gemset/gems/aws-sdk-core-2.9.44/lib/seahorse/client/base.rb:207:in `block (2 levels) in define_operation_methods'
/opt/rh/cfme-gemset/bundler/gems/cfme-providers-amazon-0c9d5e149707/app/models/manageiq/providers/amazon/inventory/collector/cloud_manager.rb:51:in `referenced_images'
/opt/rh/cfme-gemset/bundler/gems/cfme-providers-amazon-0c9d5e149707/app/models/manageiq/providers/amazon/inventory/parser/cloud_manager.rb:39:in `referenced_images'
/opt/rh/cfme-gemset/bundler/gems/cfme-providers-amazon-0c9d5e149707/app/models/manageiq/providers/amazon/inventory/parser/cloud_manager.rb:18:in `parse'
/var/www/miq/vmdb/app/models/manager_refresh/inventory.rb:38:in `block in parse'
/var/www/miq/vmdb/app/models/manager_refresh/inventory.rb:35:in `each'
/var/www/miq/vmdb/app/models/manager_refresh/inventory.rb:35:in `parse'

This is fixed upstream via this commit:
https://github.com/ManageIQ/manageiq-providers-amazon/commit/bee067db964cc488b301d6fa01b666f34c66a232

Version-Release number of selected component (if applicable):
End customer is using 5.9.3 however, this appears to not have been fixed in latest-z at this time.

How reproducible:
Every time.

Steps to Reproduce:
1. Connect to an AWS provider
2. Attempt to perform some query that would require >200 items in a filter
3. Observe error

Actual results:
AWS calls fail when large filtered results are in use

Expected results:
AWS calls should be broken up into multiple requests

Comment 2 Satoe Imaishi 2018-11-19 18:42:57 UTC

*** This bug has been marked as a duplicate of bug 1612086 ***


Note You need to log in before you can comment on or make changes to this bug.