Bug 2056884
Summary: | scp empties a file when src and dst locations are the same | |||
---|---|---|---|---|
Product: | Red Hat Enterprise Linux 9 | Reporter: | Lukáš Doktor <ldoktor> | |
Component: | openssh | Assignee: | Zoltan Fridrich <zfridric> | |
Status: | CLOSED ERRATA | QA Contact: | Marek Havrila <mhavrila> | |
Severity: | high | Docs Contact: | Jan Fiala <jafiala> | |
Priority: | urgent | |||
Version: | 9.0 | CC: | bugproxy, dbelyavs, jafiala, jjelen, kcleveng, mhavrila, mjahoda, mmatsuya, szidek, tscherf, tstaudt, zfridric | |
Target Milestone: | rc | Keywords: | Documentation, Triaged, ZStream | |
Target Release: | --- | |||
Hardware: | Unspecified | |||
OS: | Unspecified | |||
Whiteboard: | ||||
Fixed In Version: | openssh-8.7p1-12.el9 | Doc Type: | Known Issue | |
Doc Text: |
.With a specific syntax, `scp` empties files copied to themselves
The `scp` utility changed from the Secure copy protocol (SCP) to the more secure SSH file transfer protocol (SFTP). Consequently, copying a file from a location to the same location erases the file content. The problem affects the following syntax:
`scp localhost:/myfile localhost:/myfile`
To work around this problem, do not copy files to a destination that is the same as the source location using this syntax.
The problem has been fixed for the following syntaxes:
* `scp /myfile localhost:/myfile`
* `scp localhost:~/myfile ~/myfile`
|
Story Points: | --- | |
Clone Of: | ||||
: | 2108409 (view as bug list) | Environment: | ||
Last Closed: | 2022-11-15 11:21:44 UTC | Type: | Bug | |
Regression: | --- | Mount Type: | --- | |
Documentation: | --- | CRM: | ||
Verified Versions: | Category: | --- | ||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
Cloudforms Team: | --- | Target Upstream Version: | ||
Embargoed: | ||||
Bug Depends On: | ||||
Bug Blocks: | 2108409 |
Description
Lukáš Doktor
2022-02-22 09:33:26 UTC
Note, that in RHEL9, your scp is using SFTP protocol in the background, which is a bit better defined than the SCP. I would say this is expected. Doing this is at least racy or more like undefined behavior when you try to read and write one file at the same time. It worked with SCP in RHEL8 probably by pure luck. I am really not sure what intention would this use case have in the first place, but if you need to preserve this hack for some reason, please use the -O switch, which will use the SCP. Thank you for the clarification and let's hope others will find this BZ when looking for their broken scripts. As for the why are we using it, it's used by pbench to deploy testing scripts where the clients can be remote as well as local. Pbench's is_remote detection is very simple ($hostname == localhost || 127.0.0.1) but a fix based on python is coming. Unfortunately, this is an upstream bug and has no chances to be fixed soon. I think we should have to document this behaviour as an incompatibility. *** Bug 2082440 has been marked as a duplicate of this bug. *** ------- Comment From christof.schmitt.com 2022-05-06 12:42 EDT------- (In reply to comment #10) > Thank you for the clarification and let's hope others will find this BZ when > looking for their broken scripts. As for the why are we using it, it's used > by pbench to deploy testing scripts where the clients can be remote as well > as local. Pbench's is_remote detection is very simple ($hostname == > localhost || 127.0.0.1) but a fix based on python is coming. This affects IBM Spectrum Scale, and i am guessing likely other cluster solutions as well. It might not be ideal, but for a long time, scp to all cluster nodes, including the local one was possible. My concern is that this will break more systems. Working around the changed behavior can also be risky. > Unfortunately, this is an upstream bug and has no chances to be fixed soon. > I think we should have to document this behaviour as an incompatibility. > > *** Bug 2082440 has been marked as a duplicate of this bug. *** Looking at https://www.openssh.com/txt/release-8.9 |Future deprecation notice |========================= | |A near-future release of OpenSSH will switch scp(1) from using the |legacy scp/rcp protocol to using SFTP by default. So upstream OpenSSH has not switched to sftp behind scp yet. Today, this bug only affects RHEL9, so the statement that this is an "upstream bug" is not correct. Switching protocols in the scp backend is probably a good improvement, but this should never introduce a behavior change. This should be fixed before RHEL9 GA, to avoid breakage in scp users. The switch has happened in 9.0 (upstream) which is already released: https://www.openssh.com/txt/release-9.0 so the statement is correct. Unfortunately, the behavior change when switching the protocol was inevitable and major version change is a natural place to change it. If you want to preserve the old behavior, you still may enforce using the old protocol via -O key, though it is not recommended. ------- Comment From christof.schmitt.com 2022-05-06 16:46 EDT------- (In reply to comment #12) > The switch has happened in 9.0 (upstream) which is already released: > https://www.openssh.com/txt/release-9.0 > so the statement is correct. > > Unfortunately, the behavior change when switching the protocol was > inevitable and major version change is a natural place to change it. If you > want to preserve the old behavior, you still may enforce using the old > protocol via -O key, though it is not recommended. Apologies for missing OpenSSH 9.0. The change that results in losing data is very unfortunate. Luckily, we discovered this early for our product, but i would expect that other RHEL9 users will run into the same problem. One could argue that this is a "data loss" problem, that can easily be triggered. This is a ML thread about this issue: https://lists.mindrot.org/pipermail/openssh-unix-dev/2022-February/040056.html Unfortunately upstream considers it as a corner case. It maybe worth raising an issue in OpenSSH Bugzilla (https://bugzilla.mindrot.org/) ------- Comment From christof.schmitt.com 2022-05-06 16:54 EDT------- The post at https://www.redhat.com/en/blog/openssh-scp-deprecation-rhel-9-what-you-need-know documents the change, but it does not call-out that overwriting a file will now discard data. All of that is a big surprise, and any script that just calls scp has the risk of losing data. ------- Comment From christof.schmitt.com 2022-05-06 17:03 EDT------- (In reply to comment #15) > This is a ML thread about this issue: > > https://lists.mindrot.org/pipermail/openssh-unix-dev/2022-February/040056. > html > > Unfortunately upstream considers it as a corner case. It maybe worth raising > an issue in OpenSSH Bugzilla (https://bugzilla.mindrot.org/) Can Redhat open a bugzilla for OpenSSH? From IBM, we can add a comment that this is a data-loss issue. ------- Comment From christof.schmitt.com 2022-05-10 18:22 EDT------- Opened OpenSSH bugzilla: https://bugzilla.mindrot.org/show_bug.cgi?id=3431 I still think that losing data is the worst possible outcome here. Even just returning an error would be much more user-friendly, than having users have to restore their data again. ------- Comment From christof.schmitt.com 2022-05-12 12:15 EDT------- The question about using the -O parameter to fallback to the scp protocol came up. We don't see the -O parameter as a sufficient workaround: 1) The -O flag does not exist on older scp versions, so a script can longer just call scp, but it needs to start with version checks. 2) The admin can decide to create the file /etc/ssh/disable_scp to complete disable the scp protocol, which also disables this workaround. 3) Apparently the fallback to the scp protocol will also be completely removed eventually, so we could not rely on this forever. I assume that switching out the protocol behind the scp tool is the appropriate thing to do. All we are asking is to fix this one cornercase, to not silently discard data. https://bugzilla.mindrot.org/show_bug.cgi?id=3431 is the upstream fix ------- Comment From christof.schmitt.com 2022-05-13 13:59 EDT------- Thank you for pointing out the upstream fix! This is still an urgent issue for us with RHEL9. When will this fix be available in RHEL9? Could we get an ETA for our planning? Would it be possible get a RHEL9 test package of openssh with the fix applied, so that we can verify the fix? ------- Comment From seg.com 2022-07-25 17:51 EDT------- Also, note that the related RH KB article doesn't help. An actual Z stream fix is urgent and essential. We understand your reasons and are working on this bug including Z stream. When it is fixed, we will immediately let you know Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (openssh bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:8375 |