Bug 1552927

The root cause of the "NSS error -5938 (PR_END_OF_FILE_ERROR)" is clear. The FTP server has closed the connection abruptly.

However, TLS session resumption should be enabled for NSS in curl and it does not work.

I have set up a simple reproducer using vsftpd that fails for that reason (note: it closes the connection properly, there will not be the -5938 NSS error)

This is the /etc/vsftpd/vsftpd.conf I have used:

anonymous_enable=YES
local_enable=YES
write_enable=YES
local_umask=022
dirmessage_enable=YES
xferlog_enable=YES
connect_from_port_20=YES
xferlog_std_format=YES
listen=NO
listen_ipv6=YES
pam_service_name=vsftpd
userlist_enable=YES
tcp_wrappers=YES
rsa_cert_file=/etc/ssl/private/192.168.122.91.cert.pem
rsa_private_key_file=/etc/ssl/private/192.168.122.91.key.pem
ssl_enable=YES
ssl_tlsv1=NO
ssl_sslv2=NO
ssl_sslv3=NO
ssl_tlsv1_1=NO
ssl_tlsv1_2=YES
force_local_data_ssl=YES
force_local_logins_ssl=YES
require_ssl_reuse=YES
ssl_ciphers=HIGH

- You need a trusted certificate (and its private key) for the test.
- Mind the "require_ssl_reuse=YES" option. This explicitly requires a reuse from the command channel on the data channel.

When curl is used to connect to that server, it will not reuse. Verbose output follows:

 curl -v --user foobar:<left-out> --ftp-ssl ftp://192.168.122.91/
* About to connect() to 192.168.122.91 port 21 (#0)
*   Trying 192.168.122.91...
* Connected to 192.168.122.91 (192.168.122.91) port 21 (#0)
< 220 (vsFTPd 3.0.2)
> AUTH SSL
< 234 Proceed with negotiation.
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* NSS: client certificate not found (nickname not specified)
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate:
* 	subject: CN=192.168.122.91,O=Example Inc,ST=Czechia,C=CZ
* 	start date: bře 12 12:29:35 2018 GMT
* 	expire date: bře 22 12:29:35 2019 GMT
* 	common name: 192.168.122.91
* 	issuer: CN=Example Inc Intermediate CA,OU=Example Inc Certificate Authority,O=Example Inc,ST=Czechia,C=CZ
> USER foobar
< 331 Please specify the password.
> PASS <left-out>
< 230 Login successful.
> PBSZ 0
< 200 PBSZ set to 0.
> PROT P
< 200 PROT now Private.
> PWD
< 257 "/home/foobar"
* Entry path is '/home/foobar'
> EPSV
* Connect data stream passively
* ftp_perform ends with SECONDARY: 0
< 229 Entering Extended Passive Mode (|||26674|).
*   Trying 192.168.122.91...
* Connecting to 192.168.122.91 (192.168.122.91) port 26674
* Connected to 192.168.122.91 (192.168.122.91) port 21 (#0)
> TYPE A
< 200 Switching to ASCII mode.
> LIST
< 150 Here comes the directory listing.
* Maxdownload = -1
* Doing the SSL/TLS handshake on the data stream
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* NSS: client certificate not found (nickname not specified)
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate:
* 	subject: CN=192.168.122.91,O=Example Inc,ST=Czechia,C=CZ
* 	start date: bře 12 12:29:35 2018 GMT
* 	expire date: bře 22 12:29:35 2019 GMT
* 	common name: 192.168.122.91
* 	issuer: CN=Example Inc Intermediate CA,OU=Example Inc Certificate Authority,O=Example Inc,ST=Czechia,C=CZ
* Remembering we are in dir ""
< 522 SSL connection failed; session reuse required: see require_ssl_reuse option in vsftpd.conf man page
* server did not report OK, got 522
* Connection #0 to host 192.168.122.91 left intact

After a brief discussion with Kamil, I believe the problem lies in the interface between curl and NSS, and in NSS itself:

The session cache lookup is performed in ssl_LookupSID(), nss/lib/ssl/sslnonce.c:249

249 sslSessionID *
250 ssl_LookupSID(const PRIPv6Addr *addr, PRUint16 port, const char *peerID,
251               const char *urlSvrName)
252 {
...
268         if (sid->expirationTime < now) {
269             /*
270             ** This session-id timed out.
...
279         } else if (!memcmp(&sid->addr, addr, sizeof(PRIPv6Addr)) && /* server IP addr matches */
280                    (sid->port == port) &&                           /* server port matches */
281                    /* proxy (peerID) matches */
282                    (((peerID == NULL) && (sid->peerID == NULL)) ||
283                     ((peerID != NULL) && (sid->peerID != NULL) &&
284                      PORT_Strcmp(sid->peerID, peerID) == 0)) &&
285                    /* is cacheable */
286                    (sid->u.ssl3.keys.resumable) &&
287                    /* server hostname matches. */
288                    (sid->urlSvrName != NULL) &&
289                    (0 == PORT_Strcmp(urlSvrName, sid->urlSvrName))) {
290             /* Hit */
...

To have a cache hit, you need to match (among other things) also the (destination?) port number. However, this limitation may not matter for HTTPS, but it does for FTPS where the data channel will be opened on different port. The TLS spec (RFC 5246) does not impose such limitation, its a choice of NSS.

Having said that, it means that TLS session resumption is currently not possible for FTP using the NSS library, it seems.

On the other hand, curl does not pass the port number to the NSS library properly:

gdb --args curl --user foobar:<left-out> --ftp-ssl ftp://192.168.122.91/
...
(gdb) break ssl_LookupSID
(gdb) run
Breakpoint 1, ssl_LookupSID (addr=addr@entry=0x702e48, port=59884, peerID=0x6edf40 "192.168.122.91", urlSvrName=0x6df050 "192.168.122.91") at sslnonce.c:252
252	{
(gdb) info os sockets
...
192.168.122.91 35490      192.168.122.91 60649      ESTABLISHED root       INET       STREAM     
192.168.122.91 57906      192.168.122.91 21         ESTABLISHED root       INET       STREAM     
...

Those are related connection, nothing on port=59884 (where does it come from?)

(In reply to Stepan Broz from comment #6)
> To have a cache hit, you need to match (among other things) also the
> (destination?) port number. However, this limitation may not matter for
> HTTPS, but it does for FTPS where the data channel will be opened on
> different port. The TLS spec (RFC 5246) does not impose such limitation, its
> a choice of NSS.
> 
> Having said that, it means that TLS session resumption is currently not
> possible for FTP using the NSS library, it seems.

Yes, that seems to be the underlying problem.  Thank you for finding it out!

> On the other hand, curl does not pass the port number to the NSS library
> properly:

As far as I can tell, libcurl does not pass any port number(s) to NSS.

> gdb --args curl --user foobar:<left-out> --ftp-ssl ftp://192.168.122.91/
> ...
> (gdb) break ssl_LookupSID
> (gdb) run
> Breakpoint 1, ssl_LookupSID (addr=addr@entry=0x702e48, port=59884,
> peerID=0x6edf40 "192.168.122.91", urlSvrName=0x6df050 "192.168.122.91") at
> sslnonce.c:252
> 252	{
> (gdb) info os sockets
> ...
> 192.168.122.91 35490      192.168.122.91 60649      ESTABLISHED root      
> INET       STREAM     
> 192.168.122.91 57906      192.168.122.91 21         ESTABLISHED root      
> INET       STREAM     
> ...
> 
> Those are related connection, nothing on port=59884 (where does it come
> from?)

NSS obtains the port number from OS in ssl_GetPeerInfo() via getpeername().

As far as I can tell, this bug cannot be fixed in libcurl without implementing the needed functionality (reusing TLS session across connections with different port numbers) in nss first.  Therefore I am switching the component to nss.

Please do not re-open; contact me if you have any questions.