Bug 1882368

(In reply to Miroslav Hradílek from comment #0)
> Although the basic usecase of uploading to anonymous dropbox defined in
> policy works, I was not able to 1) upload with credentials.
> 
> # sosreport -o host --upload
> . . .
> Attempting upload to Red Hat Customer Portal
> Upload attempt failed: POST request returned 500: Internal Server Error
> 

Ack. It looks like the URL determination logic is too strict, I have a fix pending.

> Also there are several problems with additional upload commands:
> 
> 2) --upload-directory does not seem to do anything
> 
> # sosreport -o host --upload-url ftp://host.example.com --upload-user
> testuser --upload-pass testpass --upload-directory pub --batch
> Attempting upload to ftp://host.example.com
> Uploaded archive successfully
> 
> Directory pub did not exist, archive was uploaded to /home/testuser without
> error. When pub was created, it was still uploaded to home directory.
> 
> Looks like it always is '/incomming' from policy.
> 

Ack. I see the problem, we set the directory option, but skip over it later.

> 3) --upload-user without --upload-pass and --batch
> 
> # sosreport -o host --upload-url ftp://host.example.com --upload-user
> testuser  --batch
> . . .
> Attempting upload to ftp://host.example.com
> Upload attempt failed: local variable 'session' referenced before assignment
> 
> Good thing that password was not asked. I'd expect better error message
> though.
> 

Ack. Can present a better error message.

> 4) Manual mentions SFTP upload but it is not implemented.
> 
> This is cosmetic but nevertheless.
> 

Correct, this is not yet implemented.

> 5) --upload-url requires --upload-user
> 
> it is not inherited from policy to be 'anonymous'.
> 
> # sosreport -o host --upload-url ftp://host.example.com --batch
> Attempting upload to ftp://host.example.com
> Upload attempt failed: 530 Please login with USER and PASS.
> 

This is intentional. If you override the upload-url, we don't make any assumptions as to credentials.

> 6) I have yet to test those commands with https (https://) upload server.
> 
> Version-Release number of selected component (if applicable):
> sos-3.9.1-6.el8.noarch
> 
> Additional info:
> I used defaults for simple upload test and simple upload test with customer
> portal credentials.
> 
> I used default vsftpd instalation adding testuser to the system to verify
> functionality of: --upload-url --upload-directory --upload-user
> --upload-pass.
> 
> Note to self: To test anonymous with url on commandline, I used vsftpd with
> anonymous directives enabled in config, also had to enable boolean and give
> write access to subdirectories like /var/ftp/incomming.
> 
> 7.9 is likely affected too.



I'll have a patch up later today as these are trivial fixes.

Proposed fixes in upstream here: https://github.com/sosreport/sos/pull/2245

When this gets merged, I'll backport to the 3.x branch so this can be brought into 7.9. I'm not sure that test case 1 affects 7.9, but the others definitely do. In any event, the relaxed logic for test 1 won't hurt 7.9 at all.

Jake,
I was testing the upstream patch and found one unexpected behaviour:

_not_ setting --upload-user either --upload-pass, and running just:

python3 ./bin/sosreport -o host --upload --batch

I get:
..
Attempting upload to Red Hat Customer Portal
Upload attempt failed: Authentication failed: invalid user credentials


If no credentials are provided, shall not we fail over to uploading to the anonymous FTP?

Before your patch in #2245, sosreport in this scenario did attempt to contact ftp://dropbox.redhat.com (and failed). Now it tries the portal.

I don't get that behavior in either the 3.9 or 4.0 branches - it uploads to FTP as expected when neither a username or password is provided:

3.9:
------8<-----------
$ sudo ./sosreport --batch -o lvm2,host --upload

sosreport (version 3.9)


 Setting up archive ...
 Setting up plugins ...
 Running plugins. Please wait ...

  Starting 1/2   host            [Running: host]
  Starting 2/2   lvm2            [Running: host lvm2]
  Finishing plugins              [Running: lvm2]

  Finished running plugins

Creating compressed archive...

Your sosreport has been generated and saved in:
  /var/tmp/sosreport-terra-2020-10-09-ecjqqsy.tar.xz

Please send this file to your support representative.

Attempting upload to ftp://dropbox.redhat.com
Uploaded archive successfully
------8<-----------



4.0:

------8<-----------
$ sudo ./bin/sos report --batch -o lvm2,host --upload

sosreport (version 4.0)

 Setting up archive ...
 Setting up plugins ...
 Running plugins. Please wait ...

  Starting 1/2   host            [Running: host]
  Starting 2/2   lvm2            [Running: host lvm2]
  Finishing plugins              [Running: lvm2]

  Finished running plugins

Creating compressed archive...

Your sosreport has been generated and saved in:
	/var/tmp/sosreport-terra-2020-10-09-ppybcfo.tar.xz

Please send this file to your support representative.

Attempting upload to ftp://dropbox.redhat.com
Uploaded archive successfully

------8<-----------

For clarity - that 4.0 test is on current upstream master with #2245, and the 3.9 test does include the backport from Jan's PR.

Ah, I had some specific config that atered that behaviour.

See:

# python3 ./bin/sosreport -o host --upload --batch
..
Attempting upload to ftp://dropbox.redhat.com
Uploaded archive successfully
#

BUT:

# python3 ./bin/sosreport -o host --upload --batch --case-id=02732787
..
Attempting upload to Red Hat Customer Portal
Upload attempt failed: Authentication failed: invalid user credentials
#

case-id forces sosreport to use customer portal.

I manually tested the addressed issues from comment #2
The HTTPS protocol remain NOT tested.


Upload with credentials to Customer Portal now WORKS
# sos report -o host --upload

sosreport (version 4.0)
...
Your sosreport has been generated and saved in:
	/var/tmp/sosreport-host-10-0-138-18-02732787-2021-01-19-bdguhmz.tar.xz
...
Attempting upload to Red Hat Customer Portal
Uploaded archive successfully


Upload to FTP server using --upload-directory now WORKS
# sos report -o host --upload-url ftp://localhost --upload-user testuser --upload-pass qwerty --upload-directory /home/testuser/ftp --batch

sosreport (version 4.0)
...
Your sosreport has been generated and saved in:
	/var/tmp/sosreport-host-10-0-138-18-2021-01-19-qywvpez.tar.xz
...
Attempting upload to ftp://localhost
Uploaded archive successfully
# find /home/testuser/ftp/
/home/testuser/ftp/
/home/testuser/ftp/sosreport-host-10-0-138-18-2021-01-19-qywvpez.tar.xz


Upload to FTP server with --upload-user --batch and without --upload-pass
still prints ugly message, but that is addressed in BZ#1916729
# sos report -o host --upload-url ftp://localhost --upload-user testuser  --batch

sosreport (version 4.0)
...
Your sosreport has been generated and saved in:
	/var/tmp/sosreport-host-10-0-138-18-2021-01-19-ziwnghy.tar.xz
...
Attempting upload to ftp://localhost
Upload attempt failed: local variable 'session' referenced before assignment

Tested on RHEL-8.4.0 x86_64
sos-4.0-5.el8.noarch

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (sos bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2021:1604