* Description of problem: While using Ceph Rados Gateway with civetweb as frontend, segmented downloads of large files gets corrupted. This issue is seen only when the downloaded files are very large and multiple instances of the downloads are run in parallel. * Version-Release number of selected component (if applicable): 3.2 * How reproducible: Always * Steps to Reproduce: We can reproduce by using a test bucket, put a large file there and use 'aria2' download manager to download 10 instances of the files using 8 segments for each file and with 250 KB/s rate limit, calculating the MD5s, and comparing them. For eg: - Copy an ISO file to a test bucket # aws s3 cp CentOS-7-x86_64-NetInstall-1810.iso s3://test --acl public-read-write --endpoint-url http://10.74.255.176:8080 - Install aria2 in another server # wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm # rpm -ivvh epel-release-latest-7.noarch.rpm # yum install aria2 - Run the test command : # for i in {1..10}; do aria2c --max-overall-download-limit=250K -s 8 -x 8 -o test$i "http://10.74.255.176:8080/test/CentOS-7-x86_64-NetInstall-1810.iso" && md5sum test$i; done * Actual results: Downloaded files have different md5sums, which implies they are corrupted. * Expected results: All downloaded files should have the same md5sum. * Additional info: We tested the same thing using 'Beast' and was not able to recreate the issue.
Hello, There is a correction in the command mentioned in the description to download the files parallely. It should be : ==== for i in {1..10}; do aria2c --max-overall-download-limit=250K -s 8 -x 8 -o test$i "http://10.74.255.176:8080/test/CentOS-7-x86_64-NetInstall-1810.iso" && md5sum test$i >> testmd5 & done ==== This will download the ISO file 10 times and save as 'test1' to 'test10'. We can compare the md5sum of these downloaded files which will be in the file 'testmd5' to see if it is corrupted or not. Thanks and regards! Karun
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2019:0911