Description of problem: Copying files across filesystems using boost::filesystem::copy_file generates an "Invalid cross-device link" error and truncates the destination file. Version-Release number of selected component (if applicable): This happens with at least kernel 5.18.10 (no issue with v5.18.6) and Boost 1.76 (no issue with v1.78). Therefore, this currently affects F35 and F36. How reproducible: Here's a minimal reproducible example (with /tmp mounted as tmpfs): ``` #include <iostream> #include <fstream> #include <boost/filesystem.hpp> int main() { std::string test = "test.txt", temp = "/tmp/test.txt"; std::ofstream testf(test), tempf(temp); testf << "asdf" << std::endl; tempf << "fdsa" << std::endl; testf.close(); tempf.close(); try { boost::filesystem::copy_option option = boost::filesystem::copy_option::overwrite_if_exists; boost::filesystem::copy_file(temp, test, option); std::cout << "worked ok" << std::endl; } catch(const std::exception& e) { std::cout << "Error: " << e.what() << std::endl; } remove(temp.c_str()); return 0; } ``` then ``` $ sudo dnf install g++ boost-devel $ g++ test.cpp -o test -lboost_filesystem && ./test ``` Actual results: - Error: boost::filesystem::copy_file: Invalid cross-device link: "/tmp/test.txt", "test.txt" - The test.txt file has size 0. Expected results: - The command above should output "worked ok". - The test.txt file should contain "fdsa". Additional info: At least RStudio relies on this function, and a user got his files wiped clean due to this issue. See https://github.com/rstudio/rstudio/issues/11612
This should be reported upstream.
(In reply to Iñaki Ucar from comment #0) > std::string test = "test.txt", temp = "/tmp/test.txt"; N.B. to reproduce the error in a F36 container, use temp = "/dev/shm/test.txt" instead. /tmp is not a separate tmpfs partition in the standard f36 image, but /dev/shm is.
This comment looks relevant: #if defined(BOOST_FILESYSTEM_USE_COPY_FILE_RANGE) //! copy_file implementation that uses copy_file_range loop. Requires copy_file_range to support cross-filesystem copying. int copy_file_data_copy_file_range(int infile, int outfile, uintmax_t size)
Sigh, Boost 1.76.0 completely fails to handle the EXDEV error from copy_file_range(2). This was the upstream fix: commit 4b9052f1e0b2acf625e8247582f44acdcc78a4ce Author: Andrey Semashev Date: Tue May 18 22:53:40 2021 +0300 Fallback to read/write loop if sendfile/copy_file_range fail. Since sendfile and copy_file_range can fail for some filesystems (e.g. eCryptFS), we have to fallback to the read/write loop in copy_file implementation. Additionally, since we implement the fallback now, fallback to sendfile if copy_file_range fails with EXDEV and use copy_file_range on older kernels that don't implement it for cross-filesystem copying. This may be beneficial if copy_file_range is used within a filesystem, and is performed on a remote server NFS or CIFS). Also, it was discovered that copy_file_range can also fail with EOPNOTSUPP when it is performed on an NFSv4 filesystem and the remote server does not support COPY operation. This happens on some patched kernels in RHEL/CentOS. Lastly, to make sure the copy_file_data pointer is accessed atomically, it is now declared as an atomic value. If std::atomic is unavailable, Boost.Atomic is used. Fixes https://github.com/boostorg/filesystem/issues/184.
The fix is needed in F35 too.
I don't think calling this data loss is really accurate. If an application tries to replace file A with the content of file B, and file A is truncated to empty, you still have B. All that's lost is the original content of A, but that was supposed to be replaced anyway.
Well, it depends on the usage pattern. In the RStudio issue referenced above, it is used in a "find and replace" setting. The application copies the files with matches to /tmp, performs replacements there and finally overwrites the original files. The intermediate files are deleted and you end up with a bunch of truncated files. So that's data loss.
FEDORA-2022-9096867dad has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2022-9096867dad
(In reply to Iñaki Ucar from comment #7) > Well, it depends on the usage pattern. In the RStudio issue referenced > above, it is used in a "find and replace" setting. The application copies > the files with matches to /tmp, performs replacements there and finally > overwrites the original files. The intermediate files are deleted and you > end up with a bunch of truncated files. So that's data loss. Deleting the intermediate file if an error occurred copying it back to the original seems like a bug.
It could detect that and tell the user something like "unable to replace test.txt, the file /tmp/test.txt contains your data"
And I agree with you, but that's another discussion. The fact is that many uses and abuses exist out there.
FEDORA-2022-4b3c2b910c has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2022-4b3c2b910c
FEDORA-2022-9096867dad has been pushed to the Fedora 36 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2022-9096867dad` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-9096867dad See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2022-4b3c2b910c has been pushed to the Fedora 35 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2022-4b3c2b910c` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-4b3c2b910c See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2022-9096867dad has been pushed to the Fedora 36 stable repository. If problem still persists, please make note of it in this bug report.
FEDORA-2022-4b3c2b910c has been pushed to the Fedora 35 stable repository. If problem still persists, please make note of it in this bug report.