Bug 1653573 - Add support for uploading and downloading files to and from s3
Summary: Add support for uploading and downloading files to and from s3
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: redhat-support-lib-python
Version: 7.7-Alt
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: Pranita Ghole
QA Contact: BaseOS QE - Apps
URL:
Whiteboard:
: 1631336 (view as bug list)
Depends On:
Blocks: 1892400
TreeView+ depends on / blocked
 
Reported: 2018-11-27 07:23 UTC by Vikas Rathee
Modified: 2022-03-16 11:31 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1892400 (view as bug list)
Environment:
Last Closed: 2020-11-11 21:51:32 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 5167521 0 None None None 2020-11-10 15:16:23 UTC

Description Vikas Rathee 2018-11-27 07:23:36 UTC
Description of problem:
The attachments are being moved to s3 where we will be able to upload large files (upto 5 TB) We need to update the attachment upload and download flows in RST in order to be able to use this feature.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:
Uploaded files (<=1GB) goes to NFS and files > 1 GB goes to FTP 

Expected results:
All uploads should go to s3

Additional info:

Comment 2 Jason Edgecombe 2019-01-15 20:51:52 UTC
Is this a duplicate of BZ 1631336?

Comment 3 Chris Williams 2019-01-23 16:55:01 UTC
*** Bug 1631336 has been marked as a duplicate of this bug. ***

Comment 4 Keigo Noha 2019-03-15 07:09:14 UTC
Hi,

I think the same issue occurs to users who use Customer Portal API.
Could you consider to fix this issue a.s.a.p. Because this issue affect to our partners and users' work who put their files with RST or Customer Portal API.

Regards,
Keigo Noha

Comment 6 Emmanuel Kasper 2020-06-12 08:00:19 UTC
> Actual results:
> Uploaded files (<=1GB) goes to NFS and files > 1 GB goes to FTP 

I have see this behavior causing an attachment upload failing with mysterious errors this week.
We were uploading attachments to a support case in non interactive mode via

redhat-support-tool -c 123456 path/to_very_large_sos_report

The  upload failed because on the host where the upload was taking place, a HTTP proxy was configured, but not a FTP proxy.

I see in https://github.com/redhataccess/redhat-support-tool/blob/fc03adffe8aea4d6b14436dece14a06b4f491221/src/redhat_support_tool/plugins/add_attachment.py#L365
we're switching backends depending on the attachment size.

While I agree that this is a usability improvement for some users, for other people this behavior might break their workflow in mysterious ways, as the case I described shows.

Comment 9 Emmanuel Kasper 2020-06-16 13:20:26 UTC
How to reproduce the FTP upload issue:

- recreate a environement with HTTP only internet access, by using for instance an HTTP proxy in your LAN and disabling the default route.
To reproduce this I am setting a iptables rules which blocks all outgoing connections to port 21 (ftp)
# ip6tables -A OUTPUT -p tcp --dport 22 -j DROP
# ip6tables -A OUTPUT -p tcp --dport 22 -j DROP

- upload a file larger than 1GB
redhat-support-tool addattachment -c 123456 1521M.file

- current output 
upload fails with non informative message
Uploading 1521M.file to dropbox.redhat.com ...     failed.
ERROR: Problem encountered whilst uploading the attachment.  Please consult the Red Hat Support Tool logs for details.

log output:
2020-06-16 15:12:50,692 - redhat_support_tool.plugins.add_attachment - ERROR - ERROR: Problem encountered whilst uploading the attachment.  Please consult the Red Hat Support Tool logs for details.
2020-06-16 15:12:50,694 - redhat_support_tool.helpers.launchhelper - ERROR - local variable 'timer' referenced before assignment
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/redhat_support_lib/utils/ftphelper.py", line 103, in ftp_attachment
    passwd=config.ftp_pass)
  File "/usr/lib64/python3.6/ftplib.py", line 117, in __init__
    self.connect(host)
  File "/usr/lib64/python3.6/ftplib.py", line 152, in connect
    source_address=self.source_address)
  File "/usr/lib64/python3.6/socket.py", line 724, in create_connection
    raise err
  File "/usr/lib64/python3.6/socket.py", line 713, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out

expected output:
either one of those:
 - redhat-support-tools exits, stating than a HTTP upload of a file larger than 1GB is not possible, and suggesting the use of the --split or --use-ftp options is needed here.
 - redhat-support-tools supports HTTP uploads of file up to to 256GB like the portal itself, not swichting internally protocol depending on the uploaded file size.

Comment 16 Chris Williams 2020-11-11 21:51:32 UTC
Red Hat Enterprise Linux 7 shipped it's final minor release on September 29th, 2020. 7.9 was the last minor releases scheduled for RHEL 7.
From intial triage it does not appear the remaining Bugzillas meet the inclusion criteria for Maintenance Phase 2 and will now be closed. 

From the RHEL life cycle page:
https://access.redhat.com/support/policy/updates/errata#Maintenance_Support_2_Phase
"During Maintenance Support 2 Phase for Red Hat Enterprise Linux version 7,Red Hat defined Critical and Important impact Security Advisories (RHSAs) and selected (at Red Hat discretion) Urgent Priority Bug Fix Advisories (RHBAs) may be released as they become available."

If this BZ was closed in error and meets the above criteria please re-open it flag for 7.9.z, provide suitable business and technical justifications, and follow the process for Accelerated Fixes:
https://source.redhat.com/groups/public/pnt-cxno/pnt_customer_experience_and_operations_wiki/support_delivery_accelerated_fix_release_handbook  

Feature Requests can re-opened and moved to RHEL 8 if the desired functionality is not already present in the product. 

Please reach out to the applicable Product Experience Engineer[0] if you have any questions or concerns.  

[0] https://bugzilla.redhat.com/page.cgi?id=agile_component_mapping.html&product=Red+Hat+Enterprise+Linux+7

Comment 17 Pranita Ghole 2022-03-16 11:31:19 UTC
This has been resolved in redhat-support-tool, version 0.13.0 in the latest update to RHEL7.9 and RHEL8


Note You need to log in before you can comment on or make changes to this bug.