RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1882368 - upload functionality issues in 3.9
Summary: upload functionality issues in 3.9
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: sos
Version: 8.3
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: 8.0
Assignee: Pavel Moravec
QA Contact: Maros Kopec
URL:
Whiteboard:
Depends On:
Blocks: 1886432 1894575 1916729
TreeView+ depends on / blocked
 
Reported: 2020-09-24 11:35 UTC by Miroslav Hradílek
Modified: 2022-06-13 13:12 UTC (History)
7 users (show)

Fixed In Version: sos-4.0-2.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1886432 1916729 (view as bug list)
Environment:
Last Closed: 2021-05-18 14:48:58 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github sosreport sos pull 2245 0 None closed [policy] Fix several failure conditions with upload 2021-01-18 13:30:59 UTC
Github sosreport sos pull 2276 0 None closed [policy] Use FTP server when user isnt set in batch mode 2021-01-18 13:30:59 UTC

Internal Links: 1886432

Description Miroslav Hradílek 2020-09-24 11:35:23 UTC
Description of problem:
3.9 introduced commands for uploading sosreport archive to remote server via https and ftp. Options: --upload --upload-url --upload-directory --upload-user --upload-pass.

Although the basic usecase of uploading to anonymous dropbox defined in policy works, I was not able to 1) upload with credentials.

# sosreport -o host --upload
. . .
Attempting upload to Red Hat Customer Portal
Upload attempt failed: POST request returned 500: Internal Server Error

Also there are several problems with additional upload commands:

2) --upload-directory does not seem to do anything

# sosreport -o host --upload-url ftp://host.example.com --upload-user testuser --upload-pass testpass --upload-directory pub --batch
Attempting upload to ftp://host.example.com
Uploaded archive successfully

Directory pub did not exist, archive was uploaded to /home/testuser without error. When pub was created, it was still uploaded to home directory.

Looks like it always is '/incomming' from policy.

3) --upload-user without --upload-pass and --batch

# sosreport -o host --upload-url ftp://host.example.com --upload-user testuser  --batch
. . .
Attempting upload to ftp://host.example.com
Upload attempt failed: local variable 'session' referenced before assignment

Good thing that password was not asked. I'd expect better error message though.

4) Manual mentions SFTP upload but it is not implemented.

This is cosmetic but nevertheless.

5) --upload-url requires --upload-user

it is not inherited from policy to be 'anonymous'.

# sosreport -o host --upload-url ftp://host.example.com --batch
Attempting upload to ftp://host.example.com
Upload attempt failed: 530 Please login with USER and PASS.

6) I have yet to test those commands with https (https://) upload server.

Version-Release number of selected component (if applicable):
sos-3.9.1-6.el8.noarch

Additional info:
I used defaults for simple upload test and simple upload test with customer portal credentials.

I used default vsftpd instalation adding testuser to the system to verify functionality of: --upload-url --upload-directory --upload-user --upload-pass.

Note to self: To test anonymous with url on commandline, I used vsftpd with anonymous directives enabled in config, also had to enable boolean and give write access to subdirectories like /var/ftp/incomming.

7.9 is likely affected too.

Comment 2 Jake Hunsaker 2020-09-24 14:19:31 UTC
(In reply to Miroslav Hradílek from comment #0)
> Although the basic usecase of uploading to anonymous dropbox defined in
> policy works, I was not able to 1) upload with credentials.
> 
> # sosreport -o host --upload
> . . .
> Attempting upload to Red Hat Customer Portal
> Upload attempt failed: POST request returned 500: Internal Server Error
> 

Ack. It looks like the URL determination logic is too strict, I have a fix pending.

> Also there are several problems with additional upload commands:
> 
> 2) --upload-directory does not seem to do anything
> 
> # sosreport -o host --upload-url ftp://host.example.com --upload-user
> testuser --upload-pass testpass --upload-directory pub --batch
> Attempting upload to ftp://host.example.com
> Uploaded archive successfully
> 
> Directory pub did not exist, archive was uploaded to /home/testuser without
> error. When pub was created, it was still uploaded to home directory.
> 
> Looks like it always is '/incomming' from policy.
> 

Ack. I see the problem, we set the directory option, but skip over it later.

> 3) --upload-user without --upload-pass and --batch
> 
> # sosreport -o host --upload-url ftp://host.example.com --upload-user
> testuser  --batch
> . . .
> Attempting upload to ftp://host.example.com
> Upload attempt failed: local variable 'session' referenced before assignment
> 
> Good thing that password was not asked. I'd expect better error message
> though.
> 

Ack. Can present a better error message.

> 4) Manual mentions SFTP upload but it is not implemented.
> 
> This is cosmetic but nevertheless.
> 

Correct, this is not yet implemented.

> 5) --upload-url requires --upload-user
> 
> it is not inherited from policy to be 'anonymous'.
> 
> # sosreport -o host --upload-url ftp://host.example.com --batch
> Attempting upload to ftp://host.example.com
> Upload attempt failed: 530 Please login with USER and PASS.
> 

This is intentional. If you override the upload-url, we don't make any assumptions as to credentials.

> 6) I have yet to test those commands with https (https://) upload server.
> 
> Version-Release number of selected component (if applicable):
> sos-3.9.1-6.el8.noarch
> 
> Additional info:
> I used defaults for simple upload test and simple upload test with customer
> portal credentials.
> 
> I used default vsftpd instalation adding testuser to the system to verify
> functionality of: --upload-url --upload-directory --upload-user
> --upload-pass.
> 
> Note to self: To test anonymous with url on commandline, I used vsftpd with
> anonymous directives enabled in config, also had to enable boolean and give
> write access to subdirectories like /var/ftp/incomming.
> 
> 7.9 is likely affected too.



I'll have a patch up later today as these are trivial fixes.

Comment 3 Jake Hunsaker 2020-09-24 15:03:33 UTC
Proposed fixes in upstream here: https://github.com/sosreport/sos/pull/2245

When this gets merged, I'll backport to the 3.x branch so this can be brought into 7.9. I'm not sure that test case 1 affects 7.9, but the others definitely do. In any event, the relaxed logic for test 1 won't hurt 7.9 at all.

Comment 5 Pavel Moravec 2020-10-09 06:55:12 UTC
Jake,
I was testing the upstream patch and found one unexpected behaviour:

_not_ setting --upload-user either --upload-pass, and running just:

python3 ./bin/sosreport -o host --upload --batch

I get:
..
Attempting upload to Red Hat Customer Portal
Upload attempt failed: Authentication failed: invalid user credentials


If no credentials are provided, shall not we fail over to uploading to the anonymous FTP?

Before your patch in #2245, sosreport in this scenario did attempt to contact ftp://dropbox.redhat.com (and failed). Now it tries the portal.

Comment 6 Jake Hunsaker 2020-10-09 14:14:28 UTC
I don't get that behavior in either the 3.9 or 4.0 branches - it uploads to FTP as expected when neither a username or password is provided:

3.9:
------8<-----------
$ sudo ./sosreport --batch -o lvm2,host --upload

sosreport (version 3.9)


 Setting up archive ...
 Setting up plugins ...
 Running plugins. Please wait ...

  Starting 1/2   host            [Running: host]
  Starting 2/2   lvm2            [Running: host lvm2]
  Finishing plugins              [Running: lvm2]

  Finished running plugins

Creating compressed archive...

Your sosreport has been generated and saved in:
  /var/tmp/sosreport-terra-2020-10-09-ecjqqsy.tar.xz

Please send this file to your support representative.

Attempting upload to ftp://dropbox.redhat.com
Uploaded archive successfully
------8<-----------



4.0:

------8<-----------
$ sudo ./bin/sos report --batch -o lvm2,host --upload

sosreport (version 4.0)

 Setting up archive ...
 Setting up plugins ...
 Running plugins. Please wait ...

  Starting 1/2   host            [Running: host]
  Starting 2/2   lvm2            [Running: host lvm2]
  Finishing plugins              [Running: lvm2]

  Finished running plugins

Creating compressed archive...

Your sosreport has been generated and saved in:
	/var/tmp/sosreport-terra-2020-10-09-ppybcfo.tar.xz

Please send this file to your support representative.

Attempting upload to ftp://dropbox.redhat.com
Uploaded archive successfully

------8<-----------

Comment 7 Jake Hunsaker 2020-10-09 14:15:38 UTC
For clarity - that 4.0 test is on current upstream master with #2245, and the 3.9 test does include the backport from Jan's PR.

Comment 8 Pavel Moravec 2020-10-09 14:52:37 UTC
Ah, I had some specific config that atered that behaviour.

See:

# python3 ./bin/sosreport -o host --upload --batch
..
Attempting upload to ftp://dropbox.redhat.com
Uploaded archive successfully
#

BUT:

# python3 ./bin/sosreport -o host --upload --batch --case-id=02732787
..
Attempting upload to Red Hat Customer Portal
Upload attempt failed: Authentication failed: invalid user credentials
#

case-id forces sosreport to use customer portal.

Comment 11 Maros Kopec 2021-01-19 12:37:01 UTC
I manually tested the addressed issues from comment #2
The HTTPS protocol remain NOT tested.


Upload with credentials to Customer Portal now WORKS
# sos report -o host --upload

sosreport (version 4.0)
...
Your sosreport has been generated and saved in:
	/var/tmp/sosreport-host-10-0-138-18-02732787-2021-01-19-bdguhmz.tar.xz
...
Attempting upload to Red Hat Customer Portal
Uploaded archive successfully


Upload to FTP server using --upload-directory now WORKS
# sos report -o host --upload-url ftp://localhost --upload-user testuser --upload-pass qwerty --upload-directory /home/testuser/ftp --batch

sosreport (version 4.0)
...
Your sosreport has been generated and saved in:
	/var/tmp/sosreport-host-10-0-138-18-2021-01-19-qywvpez.tar.xz
...
Attempting upload to ftp://localhost
Uploaded archive successfully
# find /home/testuser/ftp/
/home/testuser/ftp/
/home/testuser/ftp/sosreport-host-10-0-138-18-2021-01-19-qywvpez.tar.xz


Upload to FTP server with --upload-user --batch and without --upload-pass
still prints ugly message, but that is addressed in BZ#1916729
# sos report -o host --upload-url ftp://localhost --upload-user testuser  --batch

sosreport (version 4.0)
...
Your sosreport has been generated and saved in:
	/var/tmp/sosreport-host-10-0-138-18-2021-01-19-ziwnghy.tar.xz
...
Attempting upload to ftp://localhost
Upload attempt failed: local variable 'session' referenced before assignment

Comment 12 Maros Kopec 2021-01-19 12:46:01 UTC
Tested on RHEL-8.4.0 x86_64
sos-4.0-5.el8.noarch

Comment 14 errata-xmlrpc 2021-05-18 14:48:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (sos bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2021:1604


Note You need to log in before you can comment on or make changes to this bug.