Bug 2112348

Summary: pass the OPENSSL_CONF env var through to the "ssh" binary
Product: Red Hat Enterprise Linux 9 Reporter: Laszlo Ersek <lersek>
Component: libvirtAssignee: Michal Privoznik <mprivozn>
libvirt sub component: General QA Contact: Lili Zhu <lizhu>
Status: CLOSED ERRATA Docs Contact:
Severity: unspecified    
Priority: unspecified CC: dzheng, jdenemar, lmen, mprivozn, mxie, rjones, vhodina, virt-maint, xuzhang
Version: 9.1Keywords: Triaged
Target Milestone: rcFlags: pm-rhel: mirror+
Target Release: 9.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-8.5.0-5.el9 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-11-15 10:04:47 UTC Type: Enhancement
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2062360    

Description Laszlo Ersek 2022-07-29 12:53:26 UTC
*** Description of problem:

In RHEL-9, the SHA1 algorithm is disabled for use in signature
algorithms, down at the OpenSSL level, in the non-LEGACY crypto
policies. This is a problem for virt-v2v, because virt-v2v wants to
connect to RHEL5 Xen hosts over ssh, and RHEL5 cannot use more advanced
hash algos for signatures than SHA1. For virt-v2v's sake, we don't want
to downgrade the host-wide crypto policy to LEGACY; we need a smaller
impact (connection- or invocation-specific) crypto policy relaxation.

The RHEL crypto team explained to us how this can be done, with a custom
openssl config file. The config file can be exposed to a process by
pointing the OPENSSL_CONF env variable to the config file's pathname.

We expected the "ssh" binary, started by the libvirt client library's
"remote" protocol driver, to see OPENSSL_CONF, once we set it. However,
libvirt-client actually filters this variable out, on the following call
path:

  setup                                            [virt-v2v/input/input_xen_ssh.ml]
    Libvirt.Connect.connect_auth
      ...
        remoteConnectOpen()                        [libvirt/src/remote/remote_driver.c]
          remoteSplitURIScheme()                   [libvirt/src/remote/remote_sockets.c]
            -> outputs REMOTE_DRIVER_TRANSPORT_SSH
          doRemoteOpen()                           [libvirt/src/remote/remote_driver.c]
            virNetClientNewSSH()                   [libvirtsrc/rpc/virnetclient.c]
              virNetSocketNewConnectSSH()          [libvirtsrc/rpc/virnetsocket.c]
                virCommandAddEnvPassCommon()       [libvirtsrc/util/vircommand.c]
                virCommandAddEnvPass()             [libvirtsrc/util/vircommand.c]
                virNetSocketNewConnectCommand()    [libvirtsrc/rpc/virnetsocket.c]
                  virCommandRunAsync()             [libvirtsrc/util/vircommand.c]
                    virExec()                      [libvirtsrc/util/vircommand.c]
                      execve()

In virNetSocketNewConnectSSH(), the following environment variables are
explicitly collected (copied into the "virCommand.env" array), with
virCommandAddEnvPassCommon() and virCommandAddEnvPass():

- LC_ALL (always set to "C"),
- LD_PRELOAD,
- LD_LIBRARY_PATH,
- PATH,
- HOME,
- USER,
- LOGNAME,
- TMPDIR,
- XDG_RUNTIME_DIR,
- KRB5CCNAME,
- SSH_AUTH_SOCK,
- SSH_ASKPASS,
- DISPLAY,
- XAUTHORITY.

Then in virExec(), we pass the explicitly created environment list to
execve().

OPENSSL_CONF is not on the allow-list of env variables.

Rich Jones resolved this with upstream commit 45912ac399ab ("rpc: Pass
OPENSSL_CONF through to ssh invocations", 2022-07-25); the request here
is to backport that commit to RHEL-9.1. This is necessary for us to
solve bug 2062360.

*** Version-Release number of selected component (if applicable):
libvirt-libs-8.5.0-2.el9.x86_64
libvirt-client-8.5.0-2.el9.x86_64


*** How reproducible:
100%


*** Steps to Reproduce:

1. Install a RHEL5 Xen host, and boot it with the Xen hypervisor and the
Xen dom0 kernel.

2. On the RHEL9 installation that we're going to use as the ssh and
libvirt client, create the following ~/.ssh/config stanza:

  Host 192.168.122.108
    KexAlgorithms            +diffie-hellman-group14-sha1
    MACs                     +hmac-sha1
    HostKeyAlgorithms        +ssh-rsa
    PubkeyAcceptedKeyTypes   +ssh-rsa
    PubkeyAcceptedAlgorithms +ssh-rsa

(Update the IP address as needed.)

3. On the same RHEL9 installation, create the following OpenSSL config
file, called "$HOME/openssl-sha1.cnf":

  .include /etc/ssl/openssl.cnf
  [openssl_init]
  alg_section = evp_properties
  [evp_properties]
  rh-allow-sha1-signatures = yes

(Leading whitespace should be stripped.)

4. On the RHEL9 installation, run

  update-crypto-policies --show

as root, and make sure that it not return "LEGACY". Normally it should
output "DEFAULT".

5. Run the following command on the RHEL9 installation:

$ OPENSSL_CONF=$HOME/openssl-sha1.cnf \
  virsh -c 'xen+ssh://root.122.108' list

(Update the IP address as needed.)


*** Actual results:

error: failed to connect to the hypervisor
error: Cannot recv data: ssh_dispatch_run_fatal: Connection to
192.168.122.108 port 22: error in libcrypto: Connection reset by peer


*** Expected results:

 Id   Name       State
---------------------------
 0    Domain-0   running

Comment 2 Richard W.M. Jones 2022-07-29 13:48:13 UTC
With libvirt-8.5.0-2.el9.x86_64:

$ OPENSSL_CONF=/var/tmp/openssl.cnf virsh -c 'qemu+ssh://root.0.91/system' list
error: failed to connect to the hypervisor
error: Cannot recv data: ssh: connect to host 192.168.0.91 port 22: No route to host: Connection reset by peer

With libvirt-8.5.0-5.el9_rc.03627d98b7.x86_64:

$ OPENSSL_CONF=/var/tmp/openssl.cnf virsh -c 'qemu+ssh://root.0.91/system' list
root.0.91's password: 
 Id   Name   State
--------------------


Looks like the problem is fixed with the scratch build.

Comment 3 Lili Zhu 2022-08-03 05:47:05 UTC
Tested with:
libvirt-8.5.0-3.el9.x86_64
Tested with the above build:
libvirt-8.5.0-5.el9_rc.03627d98b7.x86_64

1. On the RHEL9 installation that we're going to use as the ssh and
libvirt client, create the following ~/.ssh/config stanza:

  Host 10.73.*.*
    KexAlgorithms            +diffie-hellman-group14-sha1
    MACs                     +hmac-sha1
    HostKeyAlgorithms        +ssh-rsa
    PubkeyAcceptedKeyTypes   +ssh-rsa
    PubkeyAcceptedAlgorithms +ssh-rsa


2. On the same RHEL9 installation, create the following OpenSSL config
file, called "$HOME/openssl-sha1.cnf":

  .include /etc/ssl/openssl.cnf
  [openssl_init]
  alg_section = evp_properties
  [evp_properties]
  rh-allow-sha1-signatures = yes

3. On the RHEL9 installation, run
# update-crypto-policies --show
DEFAULT

4. Connect to the xen server
# OPENSSL_CONF=/root/openssl-sha1.cnf   virsh -c 'xen+ssh://10.73.*.*' list
error: failed to connect to the hypervisor
error: Cannot recv data: ssh_dispatch_run_fatal: Connection to 10.73.*.* port 22: error in libcrypto: Connection reset by peer

Tested with the above build:
libvirt-8.5.0-5.el9_rc.03627d98b7.x86_64
4. Connect to the xen server
# OPENSSL_CONF=/root/openssl-sha1.cnf virsh -c 'xen+ssh://10.73.**' list
The authenticity of host '10.73.*.* (10.73.*.*)' can't be established.
RSA key fingerprint is SHA256:EUNB9K3dP6Dzl4VGuy5EScDKm6XIMysvJVEWV56bYY0.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
root.*.*'s password: 
 Id   Name       State
--------------------------
 0    Domain-0   running

Comment 4 Lili Zhu 2022-08-09 09:54:35 UTC
Verify this bug with:
libvirt-8.5.0-5.el9.x86_64

1. On the RHEL9 installation that we're going to use as the ssh and
libvirt client, create the following ~/.ssh/config stanza:

  Host 10.73.*.*
    KexAlgorithms            +diffie-hellman-group14-sha1
    MACs                     +hmac-sha1
    HostKeyAlgorithms        +ssh-rsa
    PubkeyAcceptedKeyTypes   +ssh-rsa
    PubkeyAcceptedAlgorithms +ssh-rsa


2. On the same RHEL9 installation, create the following OpenSSL config
file, called "/home/lizhu/openssl-sha1.cnf":

  .include /etc/ssl/openssl.cnf
  [openssl_init]
  alg_section = evp_properties
  [evp_properties]
  rh-allow-sha1-signatures = yes

3. On the RHEL9 installation, run
$ update-crypto-policies --show
DEFAULT

4. Connect to the xen server
$ OPENSSL_CONF=/home/lizhu/openssl-sha1.cnf virsh -c 'xen+ssh://root.*.*' list
root.*.*'s password: 
 Id   Name       State
--------------------------
 0    Domain-0   running

As the testing result match with the expected result. Mark the bug as verified.

Comment 7 errata-xmlrpc 2022-11-15 10:04:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Low: libvirt security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:8003