Hide Forgot
I have a 1GB and a 5GB file on my setup of a 3 way distribute-replicate volume, I am trying to download the files using curl, and sometimes the download is not complete, original data inside the volume, root@Unbuntu:~# ls -li /mnt/swift/drep3/cont1 total 6502060 150995556 -rw------- 1 root root 1024000000 2011-07-14 22:06 1GBfile 227 -rw------- 1 root root 5632000000 2011-07-14 06:51 5GBfile 233 -rw------- 1 root root 1048576 2011-07-14 22:03 f.1 150995562 -rw------- 1 root root 1048576 2011-07-14 22:28 f.3 failure logs, root@Unbuntu:~# curl -v -O -m 3600 -H 'X-Auth-Token: AUTH_tk9a21a82029fd49cd9003e4831d8e16b1' https://10.1.12.25:443/v1/AUTH_drep3/cont1/1GBfile -k * About to connect() to 10.1.12.25 port 443 (#0) * Trying 10.1.12.25... connected * Connected to 10.1.12.25 (10.1.12.25) port 443 (#0) * successfully set certificate verify locations: * CAfile: none CApath: /etc/ssl/certs * SSLv3, TLS handshake, Client hello (1): } [data not shown] * SSLv3, TLS handshake, Server hello (2): { [data not shown] * SSLv3, TLS handshake, CERT (11): { [data not shown] * SSLv3, TLS handshake, Server finished (14): { [data not shown] * SSLv3, TLS handshake, Client key exchange (16): } [data not shown] * SSLv3, TLS change cipher, Client hello (1): } [data not shown] * SSLv3, TLS handshake, Finished (20): } [data not shown] * SSLv3, TLS change cipher, Client hello (1): { [data not shown] * SSLv3, TLS handshake, Finished (20): { [data not shown] * SSL connection using AES256-SHA * Server certificate: * subject: C=AU; ST=Some-State; O=Internet Widgits Pty Ltd * start date: 2011-07-12 06:10:01 GMT * expire date: 2011-08-11 06:10:01 GMT * SSL: unable to obtain common name from peer certificate > GET /v1/AUTH_drep3/cont1/1GBfile HTTP/1.1 > User-Agent: curl/7.19.7 (x86_64-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15 > Host: 10.1.12.25 > Accept: */* > X-Auth-Token: AUTH_tk9a21a82029fd49cd9003e4831d8e16b1 > % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0< HTTP/1.1 200 OK < Last-Modified: Fri, 15 Jul 2011 09:04:03 GMT < Etag: b5c667a723a10a3485a33263c4c2b978 < Accept-Ranges: bytes < Content-Length: 1024000000 < Content-Type: application/octet-stream < Date: Fri, 15 Jul 2011 10:52:05 GMT < { [data not shown] * Failed writing body (232 != 16384) 0 976M 0 65304 0 0 2865k 0 0:05:48 --:--:-- 0:05:48 3751k* Closing connection #0 * SSLv3, TLS alert, Client hello (1): } [data not shown] curl: (23) SSL: unable to obtain common name from peer certificate root@Unbuntu:~# ls -l total 841020 -rw-r--r-- 1 root root 49152 2011-07-14 23:52 1GBfile -rw-r--r-- 1 root root 861106176 2011-07-14 23:22 5GBfile drwxr-xr-x 4 root root 4096 2011-07-11 18:44 glusterfs drwxr-xr-x 3 root root 4096 2011-07-14 02:04 obj root@Unbuntu:~# rm -rf 1GBfile root@Unbuntu:~# ls 5GBfile glusterfs obj root@Unbuntu:~# root@Unbuntu:~# root@Unbuntu:~# curl -v -O -m 3600 -H 'X-Auth-Token: AUTH_tk9a21a82029fd49cd9003e4831d8e16b1' https://10.1.12.25:443/v1/AUTH_drep3/cont1/1GBfile -k * About to connect() to 10.1.12.25 port 443 (#0) * Trying 10.1.12.25... connected * Connected to 10.1.12.25 (10.1.12.25) port 443 (#0) * successfully set certificate verify locations: * CAfile: none CApath: /etc/ssl/certs * SSLv3, TLS handshake, Client hello (1): } [data not shown] * SSLv3, TLS handshake, Server hello (2): { [data not shown] * SSLv3, TLS handshake, CERT (11): { [data not shown] * SSLv3, TLS handshake, Server finished (14): { [data not shown] * SSLv3, TLS handshake, Client key exchange (16): } [data not shown] * SSLv3, TLS change cipher, Client hello (1): } [data not shown] * SSLv3, TLS handshake, Finished (20): } [data not shown] * SSLv3, TLS change cipher, Client hello (1): { [data not shown] * SSLv3, TLS handshake, Finished (20): { [data not shown] * SSL connection using AES256-SHA * Server certificate: * subject: C=AU; ST=Some-State; O=Internet Widgits Pty Ltd * start date: 2011-07-12 06:10:01 GMT * expire date: 2011-08-11 06:10:01 GMT * SSL: unable to obtain common name from peer certificate > GET /v1/AUTH_drep3/cont1/1GBfile HTTP/1.1 > User-Agent: curl/7.19.7 (x86_64-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15 > Host: 10.1.12.25 > Accept: */* > X-Auth-Token: AUTH_tk9a21a82029fd49cd9003e4831d8e16b1 > % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0< HTTP/1.1 200 OK < Last-Modified: Fri, 15 Jul 2011 09:04:03 GMT < Etag: b5c667a723a10a3485a33263c4c2b978 < Accept-Ranges: bytes < Content-Length: 1024000000 < Content-Type: application/octet-stream < Date: Fri, 15 Jul 2011 10:52:28 GMT < { [data not shown] * Failed writing body (232 != 16384) 0 976M 0 65304 0 0 997k 0 0:16:42 --:--:-- 0:16:42 1080k* Closing connection #0 * SSLv3, TLS alert, Client hello (1): } [data not shown] curl: (23) SSL: unable to obtain common name from peer certificate root@Unbuntu:~# root@Unbuntu:~# root@Unbuntu:~# ls 1GBfile 5GBfile glusterfs obj root@Unbuntu:~# -li No command '-li' found, did you mean: Command 'ali' from package 'nmh' (universe) Command 'ali' from package 'mailutils-mh' (universe) Command 'qli' from package 'firebird2.1-classic' (universe) Command 'qli' from package 'firebird2.0-super' (universe) Command 'qli' from package 'firebird2.1-super' (universe) Command 'qli' from package 'firebird2.0-classic' (universe) Command 'lli' from package 'llvm' (main) Command 'cli' from package 'mono-runtime' (main) Command 'li' from package 'limo' (universe) Command 'xli' from package 'xli' (universe) -li: command not found root@Unbuntu:~# ls -l total 841020 -rw-r--r-- 1 root root 49152 2011-07-14 23:52 1GBfile -rw-r--r-- 1 root root 861106176 2011-07-14 23:22 5GBfile drwxr-xr-x 4 root root 4096 2011-07-11 18:44 glusterfs drwxr-xr-x 3 root root 4096 2011-07-14 02:04 obj ############################################################################## one more instance where 1GBfile is getting downloaded and not the 5GBfile completely, root@Unbuntu:~# curl -O -m 3600 -H 'X-Auth-Token: AUTH_tk9a21a82029fd49cd9003e4831d8e16b1' https://10.1.12.25:443/v1/AUTH_drep3/cont1/1GBfile -k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 976M 100 976M 0 0 29.6M 0 0:00:32 0:00:32 --:--:-- 34.5M root@Unbuntu:~# ls -li total 1000020 342733 -rw-r--r-- 1 root root 1024000000 2011-07-14 23:56 1GBfile 440151 drwxr-xr-x 2 root root 4096 2011-07-12 20:24 dir 269425 -rw-r--r-- 1 root root 25 2011-07-14 02:57 f.1 21030 drwxr-xr-x 4 root root 4096 2011-07-11 18:44 glusterfs 18347 drwxr-xr-x 3 root root 4096 2011-07-14 02:03 obj root@Unbuntu:~# curl -O -m 3600 -H 'X-Auth-Token: AUTH_tk9a21a82029fd49cd9003e4831d8e16b1' https://10.1.12.25:443/v1/AUTH_drep3/cont1/5GBfile -k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 4 5371M 4 222M 0 0 36.9M 0 0:02:25 0:00:06 0:02:19 33.3M curl: (23) SSL: unable to obtain common name from peer certificate root@Unbuntu:~# ls -li total 1227368 342733 -rw-r--r-- 1 root root 1024000000 2011-07-14 23:56 1GBfile 346008 -rw-r--r-- 1 root root 232800256 2011-07-14 23:56 5GBfile 440151 drwxr-xr-x 2 root root 4096 2011-07-12 20:24 dir 269425 -rw-r--r-- 1 root root 25 2011-07-14 02:57 f.1 21030 drwxr-xr-x 4 root root 4096 2011-07-11 18:44 glusterfs 18347 drwxr-xr-x 3 root root 4096 2011-07-14 02:03 obj
found that with the new, I was able to download the large files. saurabh@saurabh-laptop:~$ curl -O -H 'X-Auth-Token: AUTH_tkefdfcfb323744c288f81dd0d255f0603' https://10.1.12.25:443/v1/AUTH_drep3/cont1/1GBfile -k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 976M 100 976M 0 0 42.6M 0 0:00:22 0:00:22 --:--:-- 37.3M saurabh@saurabh-laptop:~$ ls -l *GB* -rw-r--r-- 1 saurabh saurabh 1024000000 2011-07-16 13:04 1GBfile -rw-r--r-- 1 saurabh saurabh 5632000000 2011-07-16 13:02 5GBfile saurabh@saurabh-laptop:~$
Try with some other tools also like "Cyberduck".
this may be a intermittent issue, and I tried again and the download was successful, Also, via cyberduck over mac os, though it took lot of time to download, whereas via curl over linux it was quite a good speed.