Created attachment 1320914 [details]
lftp.patch

Crash would happen every time when running the reproducer command:

/usr/bin/lftp -c 'mirror -c -P -v -v -e -I *.rpm -X "/headers/" -X "/repodata/" -X "*.src.rpm" -X "/SRPMS/" -X "*-debuginfo-*.rpm" -X "/debug/" https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/rhscl/1/os/Packages /tmp/mrepo'

After making some tests, issue was fixed with the attached patch.

The patch is a backport of:
https://github.com/lavv17/lftp/commit/f7f78542f96700761ef498b9dc9787c6a6ad479a

that incorporates:
https://github.com/lavv17/lftp/commit/c8d099c69a63b92f271aa84e2bfd5d3b6f3956c5

and to compile with ssl disabled:
https://github.com/lavv17/lftp/commit/d3a83939bbb7c729247eb7013e4a79d64225197f

The only extra chunk added was adding:

+        if(!conn)
+            return 0;

to Http.h:Http::IsConnected()


  The problem, in my understanding was the way "Clone" worked, and the copy
constructor invoked in this backtrace:

Core was generated by `/usr/bin/lftp -c mirror -c -P -v -v -e -I *.rpm -X "/headers/" -X "/repodata/" -'.
Program terminated with signal 5, Trace/breakpoint trap.
#0  Http::Http (this=0xdd5f20, f=<value optimized out>) at Http.cc:116
116	Http::Http(const Http *f) : super(f)
Missing separate debuginfos, use: debuginfo-install gnutls-2.8.5-19.el6_7.x86_64 libtasn1-2.3-6.el6_5.x86_64 readline-6.0-4.el6.x86_64
(gdb) bt
#0  Http::Http (this=0xdd5f20, f=<value optimized out>) at Http.cc:116
#1  0x00007ffff1d1e4f9 in Https::Https (this=0xdd5f20, o=<value optimized out>) at Http.cc:2147
#2  0x00007ffff1d21cd9 in Https::Clone (this=0x628fc0) at Http.h:204
#3  0x00007ffff1f33815 in MirrorJob::HandleFile (this=0x629eb0, file=0xc0ee30) at MirrorJob.cc:300
#4  0x00007ffff1f366f8 in MirrorJob::Do (this=0x629eb0) at MirrorJob.cc:834
#5  0x0000003d66e29074 in SMTask::Schedule () at SMTask.cc:238
#6  0x0000003d6721a4cd in Job::WaitDone (this=0x614d00) at Job.cc:494
#7  0x0000000000405af5 in main (argc=<value optimized out>, argv=<value optimized out>) at lftp.cc:479
(gdb) p this.ssl
$1 = {ptr = 0x0, static null = <optimized out>}

the backtrace was generated from gdb (and using lftp-4.0.9-14.el6.x86_64) when
watching this address:
(gdb) p &this.ssl.ptr
$2 = (lftp_ssl_gnutls **) 0xdd62d0

and just when it did become null:

(gdb) disassemble 
Dump of assembler code for function Http::Http(Http const*):
   0x00007ffff1d1e3f0 <+0>:	push   %rbx
   0x00007ffff1d1e3f1 <+1>:	mov    %rdi,%rbx
   0x00007ffff1d1e3f4 <+4>:	callq  0x7ffff1d1ae20 <_ZN9NetAccessC2EPKS_@plt>
   0x00007ffff1d1e3f9 <+9>:	mov    0x20a220(%rip),%rax        # 0x7ffff1f28620
   0x00007ffff1d1e400 <+16>:	mov    %rbx,%rdi
   0x00007ffff1d1e403 <+19>:	movq   $0x0,0x370(%rbx)
   0x00007ffff1d1e40e <+30>:	movq   $0x0,0x378(%rbx)
   0x00007ffff1d1e419 <+41>:	movq   $0x0,0x390(%rbx)
   0x00007ffff1d1e424 <+52>:	movq   $0x0,0x3a0(%rbx)
   0x00007ffff1d1e42f <+63>:	movq   $0x0,0x398(%rbx)
   0x00007ffff1d1e43a <+74>:	lea    0x10(%rax),%rdx
   0x00007ffff1d1e43e <+78>:	add    $0x170,%rax
   0x00007ffff1d1e444 <+84>:	movq   $0x0,0x3b0(%rbx)
=> 0x00007ffff1d1e44f <+95>:	mov    %rax,0x20(%rbx)
   0x00007ffff1d1e453 <+99>:	movq   $0x0,0x3b8(%rbx)
   0x00007ffff1d1e45e <+110>:	mov    %rdx,(%rbx)
   0x00007ffff1d1e461 <+113>:	movq   $0x0,0x3c8(%rbx)
   0x00007ffff1d1e46c <+124>:	movq   $0x0,0x3c0(%rbx)
   0x00007ffff1d1e477 <+135>:	movq   $0x0,0x3d8(%rbx)
   0x00007ffff1d1e482 <+146>:	movq   $0x0,0x3e8(%rbx)
   0x00007ffff1d1e48d <+157>:	movq   $0x0,0x3e0(%rbx)
   0x00007ffff1d1e498 <+168>:	callq  0x7ffff1d1b230 <_ZN4Http4InitEv@plt>
   0x00007ffff1d1e49d <+173>:	mov    %rbx,%rdi
   0x00007ffff1d1e4a0 <+176>:	xor    %esi,%esi
   0x00007ffff1d1e4a2 <+178>:	pop    %rbx
   0x00007ffff1d1e4a3 <+179>:	jmpq   0x7ffff1d1aeb0 <_ZN4Http8ReconfigEPKc@plt>
End of assembler dump.

(gdb) p/x $rbx+0x3b0
$3 = 0xdd62d0

this should happen at the end of the "super(f)" code, and
when using the Http::Connection subclass, the problem could
not be reproduced.