RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1722229 - scp protocol error: filename does not match request causing Oracle 18c Grid to not install
Summary: scp protocol error: filename does not match request causing Oracle 18c Grid t...
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: openssh
Version: 8.1
Hardware: x86_64
OS: Linux
low
urgent
Target Milestone: rc
: 8.0
Assignee: Jakub Jelen
QA Contact: BaseOS QE Security Team
URL:
Whiteboard:
Depends On:
Blocks: 1708794
TreeView+ depends on / blocked
 
Reported: 2019-06-19 18:10 UTC by Daniel Yeisley
Modified: 2021-01-05 16:41 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-01-05 16:41:33 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Description Daniel Yeisley 2019-06-19 18:10:56 UTC 
Description of problem:
The Oracle 18c Grid installer claims that passwordless ssh isn't setup between nodes when in fact it is. 

Version-Release number of selected component (if applicable):
openssh-server-8.0p1-1.el8.x86_64
openssh-8.0p1-1.el8.x86_64
openssh-clients-8.0p1-1.el8.x86_64

How reproducible:
Every time I've tried. 

Steps to Reproduce:
1. Setup passwordless ssh between nodes with ssh-keygen and ssh-copy-id.
2. Attempt to install Oracle 18c Grid.
3.

Actual results:
I looked at the debug output and found where it's failing.

[Worker 1] [ 2019-06-19 11:14:54.596 EDT ] [VerificationUtil.getDestLoc:4696]  ==== CV_DESTLOC(pre-fetched value): '/tmp/GridSetupActions2019-06-19_11-12-14AM/'
[Worker 1] [ 2019-06-19 11:14:54.597 EDT ] [UnixSystem.copyFile:1097]  Copy file veritas7:/tmp/GridSetupActions2019-06-19_11-12-14AM/CVU_18.0.0.0.0_oracle/scratch/getFileInfo27877.out to localnode:/tmp/GridSetupActions2019-06-19_11-12-14AM/veritas7.getFileInfo27877.out
[Worker 1] [ 2019-06-19 11:14:54.597 EDT ] [Utils.getLocalHost:479]  Hostname retrieved: veritas6.6a2m.lab.eng.bos.redhat.com, returned: veritas6
[Worker 1] [ 2019-06-19 11:14:54.597 EDT ] [UnixSystem.remoteCopyFile:821]  Copying files veritas7/tmp/GridSetupActions2019-06-19_11-12-14AM/CVU_18.0.0.0.0_oracle/scratch/getFileInfo27877.out localnode/tmp/GridSetupActions2019-06-19_11-12-14AM/veritas7.getFileInfo27877.out
[Worker 1] [ 2019-06-19 11:14:54.597 EDT ] [Utils.getLocalHost:479]  Hostname retrieved: veritas6.6a2m.lab.eng.bos.redhat.com, returned: veritas6
[Worker 1] [ 2019-06-19 11:14:54.597 EDT ] [Utils.getLocalHost:479]  Hostname retrieved: veritas6.6a2m.lab.eng.bos.redhat.com, returned: veritas6
[Worker 1] [ 2019-06-19 11:14:54.597 EDT ] [UnixSystem.remoteCopyFile:839]  UnixSystem: /usr/bin/scp -p veritas7:'/tmp/GridSetupActions2019-06-19_11-12-14AM/CVU_18.0.0.0.0_oracle/scratch/getFileInfo27877.out' /tmp/GridSetupActions2019-06-19_11-12-14AM/veritas7.getFileInfo27877.out
[Thread-432] [ 2019-06-19 11:14:54.598 EDT ] [StreamReader.run:62]  In StreamReader.run
[Worker 1] [ 2019-06-19 11:14:54.598 EDT ] [RuntimeExec.runCommand:294]  runCommand: Waiting for the process
[Thread-431] [ 2019-06-19 11:14:54.598 EDT ] [StreamReader.run:62]  In StreamReader.run
[Thread-432] [ 2019-06-19 11:14:55.184 EDT ] [StreamReader.run:66]  ERROR>protocol error: filename does not match request
[Worker 1] [ 2019-06-19 11:14:55.185 EDT ] [RuntimeExec.runCommand:296]  runCommand: process returns 1
[Worker 1] [ 2019-06-19 11:14:55.187 EDT ] [RuntimeExec.runCommand:323]  RunTimeExec: error>
[Worker 1] [ 2019-06-19 11:14:55.187 EDT ] [RuntimeExec.runCommand:326]  protocol error: filename does not match request
[Worker 1] [ 2019-06-19 11:14:55.187 EDT ] [RuntimeExec.traceCmdEnv:516]  Calling Runtime.exec() with the command
[Worker 1] [ 2019-06-19 11:14:55.187 EDT ] [RuntimeExec.traceCmdEnv:518]  /usr/bin/scp
[Worker 1] [ 2019-06-19 11:14:55.187 EDT ] [RuntimeExec.traceCmdEnv:518]  -p
[Worker 1] [ 2019-06-19 11:14:55.187 EDT ] [RuntimeExec.traceCmdEnv:518]  veritas7:'/tmp/GridSetupActions2019-06-19_11-12-14AM/CVU_18.0.0.0.0_oracle/scratch/getFileInfo27877.out'
[Worker 1] [ 2019-06-19 11:14:55.187 EDT ] [RuntimeExec.traceCmdEnv:518]  /tmp/GridSetupActions2019-06-19_11-12-14AM/veritas7.getFileInfo27877.out
[Worker 1] [ 2019-06-19 11:14:55.187 EDT ] [RuntimeExec.runCommand:349]  Returning from RunTimeExec.runCommand
[Worker 1] [ 2019-06-19 11:14:55.187 EDT ] [NativeSystem.rununixcmd:1324]  NativeSystem.rununixcmd: RetString 0|protocol error: filename does not match request :failed


Expected results:
I downgraded to the following packages and attempted to install again.
openssh-7.8p1-4.el8.x86_64.rpm  
openssh-clients-7.8p1-4.el8.x86_64.rpm  
openssh-server-7.8p1-4.el8.x86_64.rpm

From the debug output:
[Worker 2] [ 2019-06-19 13:28:00.169 EDT ] [VerificationUtil.getDestLoc:4696]  ==== CV_DESTLOC(pre-fetched value): '/tmp/GridSetupActions2019-06-19_01-24-29PM/'
[Worker 2] [ 2019-06-19 13:28:00.170 EDT ] [UnixSystem.copyFile:1097]  Copy file veritas7:/tmp/GridSetupActions2019-06-19_01-24-29PM/CVU_18.0.0.0.0_oracle/scratch/getFileInfo8471.out to localnode:/tmp/GridSetupActions2019-06-19_01-24-29PM/veritas7.getFileInfo8471.out
[Worker 2] [ 2019-06-19 13:28:00.170 EDT ] [Utils.getLocalHost:479]  Hostname retrieved: veritas6.6a2m.lab.eng.bos.redhat.com, returned: veritas6
[Worker 2] [ 2019-06-19 13:28:00.170 EDT ] [UnixSystem.remoteCopyFile:821]  Copying files veritas7/tmp/GridSetupActions2019-06-19_01-24-29PM/CVU_18.0.0.0.0_oracle/scratch/getFileInfo8471.out localnode/tmp/GridSetupActions2019-06-19_01-24-29PM/veritas7.getFileInfo8471.out
[Worker 2] [ 2019-06-19 13:28:00.170 EDT ] [Utils.getLocalHost:479]  Hostname retrieved: veritas6.6a2m.lab.eng.bos.redhat.com, returned: veritas6
[Worker 2] [ 2019-06-19 13:28:00.170 EDT ] [Utils.getLocalHost:479]  Hostname retrieved: veritas6.6a2m.lab.eng.bos.redhat.com, returned: veritas6
[Worker 2] [ 2019-06-19 13:28:00.170 EDT ] [UnixSystem.remoteCopyFile:839]  UnixSystem: /usr/bin/scp -p veritas7:'/tmp/GridSetupActions2019-06-19_01-24-29PM/CVU_18.0.0.0.0_oracle/scratch/getFileInfo8471.out' /tmp/GridSetupActions2019-06-19_01-24-29PM/veritas7.getFileInfo8471.out
[Thread-432] [ 2019-06-19 13:28:00.171 EDT ] [StreamReader.run:62]  In StreamReader.run
[Worker 2] [ 2019-06-19 13:28:00.171 EDT ] [RuntimeExec.runCommand:294]  runCommand: Waiting for the process
[Thread-431] [ 2019-06-19 13:28:00.171 EDT ] [StreamReader.run:62]  In StreamReader.run
[Worker 2] [ 2019-06-19 13:28:00.747 EDT ] [RuntimeExec.runCommand:296]  runCommand: process returns 0
[Worker 2] [ 2019-06-19 13:28:00.747 EDT ] [RuntimeExec.runCommand:323]  RunTimeExec: error>
[Worker 2] [ 2019-06-19 13:28:00.747 EDT ] [RuntimeExec.runCommand:349]  Returning from RunTimeExec.runCommand
[Worker 2] [ 2019-06-19 13:28:00.747 EDT ] [NativeSystem.isCmdScv:580]  isCmdScv: cmd=[/usr/bin/scp -p veritas7:'/tmp/GridSetupActions2019-06-19_01-24-29PM/CVU_18.0.0.0.0_oracle/scratch/getFileInfo8471.out' /tmp/GridSetupActions2019-06-19_01-24-29PM/veritas7.getFileInfo8471.out]
[Worker 2] [ 2019-06-19 13:28:00.747 EDT ] [NativeSystem.isCmdScv:630]  isCmdScv: /usr/bin/scp is present.
[Worker 2] [ 2019-06-19 13:28:00.747 EDT ] [NativeSystem.isCmdScv:632]  isCmdScv: /usr/bin/scp is a file.
[Worker 2] [ 2019-06-19 13:28:00.748 EDT ] [NativeSystem.isCmdScv:649]  isCmdScv: returned true
[Worker 2] [ 2019-06-19 13:28:00.748 EDT ] [NativeSystem.rununixcmd:1324]  NativeSystem.rununixcmd: RetString 1| :successful
[Worker 2] [ 2019-06-19 13:28:00.748 EDT ] [GetFileInfoCommand.processOutputFile:204]  <FILE><NAME,/home/oracle/.ssh/id_rsa><STATUS,0><USER,oracle><GROUP,oinstall><PERMISSIONS,0600></FILE>
[Worker 2] [ 2019-06-19 13:28:00.748 EDT ] [Utils.getLocalHost:479]  Hostname retrieved: veritas6.6a2m.lab.eng.bos.redhat.com, returned: veritas6
[Worker 2] [ 2019-06-19 13:28:00.749 EDT ] [RuntimeExec.runCommand:294]  runCommand: Waiting for the process
[Thread-434] [ 2019-06-19 13:28:00.750 EDT ] [StreamReader.run:62]  In StreamReader.run
[Thread-433] [ 2019-06-19 13:28:00.750 EDT ] [StreamReader.run:62]  In StreamReader.run
[Worker 2] [ 2019-06-19 13:28:01.319 EDT ] [RuntimeExec.runCommand:296]  runCommand: process returns 0


Additional info:
Comment 1 Daniel Yeisley 2019-06-19 18:28:10 UTC 
I don't have a problem from the command line. I'm not sure what the installer is doing differently. 

[root@veritas6 ~]# /usr/bin/scp -p veritas7:'/tmp/GridSetupActions2019-06-19_02-13-10PM/CVU_18.0.0.0.0_oracle/scratch/getFileInfo6801.out' /tmp/GridSetupActions2019-06-19_02-13-10PM/veritas7.getFileInfo6801.out
getFileInfo6801.out                                                                                                                                                              100%  102   149.7KB/s   00:00
Comment 2 Jakub Jelen 2019-06-20 07:53:34 UTC 
I think this is caused by the fix for CVE-2019-6111 [1]. In this case, the problem are the apostrophes around the filaname. You should be able to workaround this by using -T switch to skip the strict checks (see openssh release notes [2]) or use SFTP in batch mode instead (preferred).

This is indeed a change in a behavior of a legacy tool, but unfortunately nothing we can change or fix.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1666127
[2] http://www.openssh.com/txt/release-8.0
Comment 3 Daniel Yeisley 2019-06-20 17:40:57 UTC 
(In reply to Jakub Jelen from comment #2)
> I think this is caused by the fix for CVE-2019-6111 [1]. In this case, the
> problem are the apostrophes around the filaname. You should be able to
> workaround this by using -T switch to skip the strict checks (see openssh
> release notes [2]) or use SFTP in batch mode instead (preferred).
> 
> This is indeed a change in a behavior of a legacy tool, but unfortunately
> nothing we can change or fix.
> 
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1666127
> [2] http://www.openssh.com/txt/release-8.0

I do not have the ability to modify Oracle's install code. The best I can do is set an environment variable to tell the installer what scp to use. So I created a script that calls 'scp -T' and pointed the installer at that.


[oracle@veritas6 grid]$ export ORACLE_SRVM_REMOTECOPY=/grid/scp
[oracle@veritas6 grid]$ cat /grid/scp 
#!/bin/sh

/usr/bin/scp -T $*

Adding the '-T' option to the environment variable doesn't work. I tried export ORACLE_SRVM_REMOTECOPY="/grid/scp -T" and the installer bailed out right away.
Comment 4 Jakub Jelen 2019-06-21 07:11:26 UTC 
Can you show the debug log with the /grid/scp set up? Is it different than the previous attempts?
Comment 5 Daniel Yeisley 2019-06-21 16:01:48 UTC 
(In reply to Jakub Jelen from comment #4)
> Can you show the debug log with the /grid/scp set up? Is it different than
> the previous attempts?

I put my scp script in /tmp. It looks like the successful output. 

[Worker 2] [ 2019-06-21 11:48:52.588 EDT ] [VerificationUtil.getDestLoc:4696]  ==== CV_DESTLOC(pre-fetched value): '/tmp/GridSetupActions2019-06-21_11-44-06AM/'
[Worker 2] [ 2019-06-21 11:48:52.588 EDT ] [UnixSystem.copyFile:1097]  Copy file veritas7:/tmp/GridSetupActions2019-06-21_11-44-06AM/CVU_18.0.0.0.0_oracle/scratch/getFileInfo31138.out to localnode:/tmp/GridSetupActions2019-06-21_11-44-06AM/veritas7.getFileInfo31138.out
[Worker 2] [ 2019-06-21 11:48:52.588 EDT ] [Utils.getLocalHost:479]  Hostname retrieved: veritas6.6a2m.lab.eng.bos.redhat.com, returned: veritas6
[Worker 2] [ 2019-06-21 11:48:52.588 EDT ] [UnixSystem.remoteCopyFile:821]  Copying files veritas7/tmp/GridSetupActions2019-06-21_11-44-06AM/CVU_18.0.0.0.0_oracle/scratch/getFileInfo31138.out localnode/tmp/GridSetupActions2019-06-21_11-44-06AM/veritas7.getFileInfo31138.out
[Worker 2] [ 2019-06-21 11:48:52.588 EDT ] [Utils.getLocalHost:479]  Hostname retrieved: veritas6.6a2m.lab.eng.bos.redhat.com, returned: veritas6
[Worker 2] [ 2019-06-21 11:48:52.588 EDT ] [Utils.getLocalHost:479]  Hostname retrieved: veritas6.6a2m.lab.eng.bos.redhat.com, returned: veritas6
[Worker 2] [ 2019-06-21 11:48:52.588 EDT ] [UnixSystem.remoteCopyFile:839]  UnixSystem: /tmp/scp -p veritas7:'/tmp/GridSetupActions2019-06-21_11-44-06AM/CVU_18.0.0.0.0_oracle/scratch/getFileInfo31138.out' /tmp/GridSetupActions2019-06-21_11-44-06AM/veritas7.getFileInfo31138.out
[Thread-432] [ 2019-06-21 11:48:52.589 EDT ] [StreamReader.run:62]  In StreamReader.run
[Worker 2] [ 2019-06-21 11:48:52.589 EDT ] [RuntimeExec.runCommand:294]  runCommand: Waiting for the process
[Thread-431] [ 2019-06-21 11:48:52.589 EDT ] [StreamReader.run:62]  In StreamReader.run
[Worker 2] [ 2019-06-21 11:48:53.192 EDT ] [RuntimeExec.runCommand:296]  runCommand: process returns 0
[Worker 2] [ 2019-06-21 11:48:53.193 EDT ] [RuntimeExec.runCommand:323]  RunTimeExec: error>
[Worker 2] [ 2019-06-21 11:48:53.193 EDT ] [RuntimeExec.runCommand:349]  Returning from RunTimeExec.runCommand
[Worker 2] [ 2019-06-21 11:48:53.193 EDT ] [NativeSystem.isCmdScv:580]  isCmdScv: cmd=[/tmp/scp -p veritas7:'/tmp/GridSetupActions2019-06-21_11-44-06AM/CVU_18.0.0.0.0_oracle/scratch/getFileInfo31138.out' /tmp/GridSetupActions2019-06-21_11-44-06AM/veritas7.getFileInfo31138.out]
[Worker 2] [ 2019-06-21 11:48:53.193 EDT ] [NativeSystem.isCmdScv:630]  isCmdScv: /tmp/scp is present.
[Worker 2] [ 2019-06-21 11:48:53.193 EDT ] [NativeSystem.isCmdScv:632]  isCmdScv: /tmp/scp is a file.
[Worker 2] [ 2019-06-21 11:48:53.193 EDT ] [NativeSystem.isCmdScv:649]  isCmdScv: returned true
[Worker 2] [ 2019-06-21 11:48:53.193 EDT ] [NativeSystem.rununixcmd:1324]  NativeSystem.rununixcmd: RetString 1| :successful
[Worker 2] [ 2019-06-21 11:48:53.193 EDT ] [GetFileInfoCommand.processOutputFile:204]  <FILE><NAME,/home/oracle/.ssh/id_rsa><STATUS,0><USER,oracle><GROUP,oinstall><PERMISSIONS,0600></FILE>
[Worker 2] [ 2019-06-21 11:48:53.193 EDT ] [Utils.getLocalHost:479]  Hostname retrieved: veritas6.6a2m.lab.eng.bos.redhat.com, returned: veritas6
[Thread-434] [ 2019-06-21 11:48:53.194 EDT ] [StreamReader.run:62]  In StreamReader.run
[Worker 2] [ 2019-06-21 11:48:53.195 EDT ] [RuntimeExec.runCommand:294]  runCommand: Waiting for the process
[Thread-433] [ 2019-06-21 11:48:53.195 EDT ] [StreamReader.run:62]  In StreamReader.run
[Worker 2] [ 2019-06-21 11:48:53.780 EDT ] [RuntimeExec.runCommand:296]  runCommand: process returns 0
[Worker 2] [ 2019-06-21 11:48:53.780 EDT ] [RuntimeExec.runCommand:323]  RunTimeExec: error>
[Worker 2] [ 2019-06-21 11:48:53.780 EDT ] [RuntimeExec.runCommand:349]  Returning from RunTimeExec.runCommand

[root@veritas6 Certification]# rpm -qa | grep openssh
openssh-server-8.0p1-1.el8.x86_64
openssh-clients-8.0p1-1.el8.x86_64
openssh-8.0p1-1.el8.x86_64

[root@veritas6 Certification]# cat /tmp/scp 
#!/bin/sh

/usr/bin/scp -T $*
Comment 6 Jakub Jelen 2019-06-24 08:23:38 UTC 
Nice. Thank you for the verification. I understand that this is not an ideal solution, but it is best we have. Is this something you can live with for now or something that can be addressed on oracle side in future?

I could propose to store the script rather in /usr/bin/insecure-scp or something like that for the installer (not sure whether it is needed later in the process or only in the installer).
Comment 7 Daniel Yeisley 2019-06-24 12:39:06 UTC 
(In reply to Jakub Jelen from comment #6)
> Nice. Thank you for the verification. I understand that this is not an ideal
> solution, but it is best we have. Is this something you can live with for
> now or something that can be addressed on oracle side in future?
> 
> I could propose to store the script rather in /usr/bin/insecure-scp or
> something like that for the installer (not sure whether it is needed later
> in the process or only in the installer).

I can live with it. Ideally Oracle would update their code, but I have no idea how long that will take. Creating /usr/bin/insecure-scp sounds good to me.
Comment 8 Tomas Mraz 2019-06-24 14:07:14 UTC 
I probably would not put it in the directory on $PATH. Maybe /usr/libexec/scp-T would be better.
Comment 9 Tomas Mraz 2019-06-24 14:08:18 UTC 
With some appropriate comment in the script.
Comment 10 Daniel Yeisley 2019-07-02 13:09:10 UTC 
(In reply to Jakub Jelen from comment #6)
> Nice. Thank you for the verification. I understand that this is not an ideal
> solution, but it is best we have. Is this something you can live with for
> now or something that can be addressed on oracle side in future?
> 
> I could propose to store the script rather in /usr/bin/insecure-scp or
> something like that for the installer (not sure whether it is needed later
> in the process or only in the installer).

So what's happening with this?
Comment 11 Jakub Jelen 2019-07-02 13:51:37 UTC 
I am sorry if it was not clear from the last comment.

I do not think it is a good idea to provide this script from the base openssh package and teach people to use it without thinking so the previous comments were mainly addressing your installer scripts, that could create this workaround file as a short-term solution and fixing the oracle installer as a long-term solution.

I did not close this bug so far to keep it visible if somebody else will encounter same issue to serve as a landing page and simple way to find a workaround and a root cause.

Does it make sense?
Comment 12 Daniel Yeisley 2019-07-02 14:11:43 UTC 
(In reply to Jakub Jelen from comment #11)
> I am sorry if it was not clear from the last comment.
> 
> I do not think it is a good idea to provide this script from the base
> openssh package and teach people to use it without thinking so the previous
> comments were mainly addressing your installer scripts, that could create
> this workaround file as a short-term solution and fixing the oracle
> installer as a long-term solution.
> 
> I did not close this bug so far to keep it visible if somebody else will
> encounter same issue to serve as a landing page and simple way to find a
> workaround and a root cause.
> 
> Does it make sense?

So we fix this with documentation? I don't have a problem creating a simple script to work around the issue.
Comment 13 Jakub Jelen 2019-07-02 15:11:00 UTC 
Yes, we certainly plan to mention something along the lines in the RHEL 8.1 release notes because of the rebase and this CVE fix, which is causing this behavior.
Comment 16 Jakub Jelen 2021-01-05 16:41:33 UTC 
We kept this bug open for some time after RHEL8.0 if others would experience the same issue. This was documented in Release notes under section "OpenSSH rebased to 8.0p1":

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/8.1_release_notes/rhel-8_1_0_release

We do not plan to fix/change this anyhow so I am closing the bug now.
Note You need to log in before you can comment on or make changes to this bug.