Bug 962499

Thanks for a bug report. Could you send me such message, please? My gpg key ID is F3C36A0D, and is available here:
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xFB183E7EF3C36A0D

Thanks in advance.

Summary form our IRC tests:
- I can see David's message without any issue
- he can see my (encrypted & signed) message reply too
- same as if I set evo to encrypt with my own key as well (which was the case for his initial email)

Thus this might be, because the message was generated by Enigmail (version currently unknown).

Sender's environment:
   gnupg-1.4.13-2.fc18.x86_64
   thunderbird-17.0.5-1.fc18.x86_64
   enigmail 1.5.1

He sent me signed and encrypted email, and evolution's git master suffers of the same. I'm moving this to evolution-data-server, where the code resides.

One more observation: When the messages are Saved As and decrypted using gpg, the original Sender's message decryption ends up with return code 2 while Milan's message is decrypted with return code 0:

$ LC_ALL=C gpg -v -d Plocha/Fwd\:_please_send_gpg-signed___encrypted_message_to_mcrha > /dev/null ; echo $?
Charset: UTF-8
gpg: armor header: 
Version: GnuPG v1.4.13 (GNU/Linux)
gpg: armor header: 
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
gpg: armor header: 
gpg: public key is 14B339C8
gpg: public key is B8318982
gpg: using subkey B8318982 instead of primary key 22C33E24

You need a passphrase to unlock the secret key for
user: "David Jaša <djasa>"
gpg: using subkey B8318982 instead of primary key 22C33E24
4096-bit RSA key, ID B8318982, created 2011-02-16 (main key ID 22C33E24)

gpg: public key is 1561DCD6
gpg: encrypted with RSA key, ID 1561DCD6
gpg: encrypted with ELG key, ID 14B339C8
gpg: encrypted with 4096-bit RSA key, ID B8318982, created 2011-02-16
      "David Jaša <djasa>"
gpg: AES256 encrypted data
gpg: original file name=''
gpg: Signature made Wed May 15 01:17:34 2013 CEST using RSA key ID B15B82F5
gpg: Can't check signature: No public key
2


$ LC_ALL=C gpg -v -d Plocha/Re\:_gpg_d_message_test2 > /dev/null ; echo $?
Version: GnuPG v2.0.14 (GNU/Linux)
gpg: armor header: 
gpg: public key is B8318982
gpg: using subkey B8318982 instead of primary key 22C33E24

You need a passphrase to unlock the secret key for
user: "David Jaša <djasa>"
gpg: using subkey B8318982 instead of primary key 22C33E24
4096-bit RSA key, ID B8318982, created 2011-02-16 (main key ID 22C33E24)

gpg: public key is 14B339C8
gpg: encrypted with ELG key, ID 14B339C8
gpg: encrypted with 4096-bit RSA key, ID B8318982, created 2011-02-16
      "David Jaša <djasa>"
gpg: AES256 encrypted data
gpg: original file name=''
0


So the actual bug may be in gpg or tb/enigmail and fix on evolution side would be then just a workaround. (I'm not saying that it isn't worth fixing on Evo side at all but that may just cover actual bug somewhere else)

From 'man gpg' on Fedora 18:
> The program returns 0 if everything was fine, 1 if at least a signature
> was bad, and other error codes for fatal errors.

I see the difference. Enigmail creates an encrypted message which is also signed, in once with encryption, while evolution signs a message, and then encrypts this message part separately, thus the signature in evolution's message case is not tested in once with decryption.

I ran gpg2 under gdb, and I see in enigmail's message that the log_error() is called with the signature, thus at the end the gpg2 recognizes one error, and returns 2, instead of 0, despite it didn't find any error on decryption itself.

Breakpoint 2, log_error (fmt=0x489bfd "Can't check signature: %s\n") at logging.c:559
559	{
(gdb) bt
#0  log_error (fmt=0x489bfd "Can't check signature: %s\n") at logging.c:559
#1  0x000000000041d544 in check_sig_and_print (c=c@entry=0x6c1930, node=node@entry=0x6c1bd0) at mainproc.c:1973
#2  0x000000000041dc0b in proc_tree (node=<optimized out>, c=0x6c1930) at mainproc.c:2062
#3  release_list (c=c@entry=0x6c1930) at mainproc.c:114
#4  0x000000000041e25e in do_proc_packets (c=c@entry=0x6c1930, a=a@entry=0x6b4b60) at mainproc.c:1397
#5  0x000000000041e479 in proc_packets (anchor=anchor@entry=0x6c1540, a=0x6b4b60) at mainproc.c:1173
#6  0x000000000040ff5c in handle_compressed (procctx=procctx@entry=0x6c1540, cd=0x6c3040, callback=callback@entry=0x0, passthru=passthru@entry=0x0) at compress.c:312
#7  0x000000000041b81b in proc_compressed (c=c@entry=0x6c1540, pkt=pkt@entry=0x6c1610) at mainproc.c:781
#8  0x000000000041e193 in do_proc_packets (c=c@entry=0x6c1540, a=a@entry=0x6b4b60) at mainproc.c:1361
#9  0x000000000041e479 in proc_packets (anchor=anchor@entry=0x6bced0, a=0x6b4b60) at mainproc.c:1173
#10 0x000000000043a163 in decrypt_data (procctx=procctx@entry=0x6bced0, ed=0x6b4870, dek=0x7ffff7ff5008) at encr-data.c:199
#11 0x000000000041af3a in proc_encrypted (c=c@entry=0x6bced0, pkt=pkt@entry=0x6b4cd0) at mainproc.c:562
#12 0x000000000041e1ab in do_proc_packets (c=c@entry=0x6bced0, a=a@entry=0x6b4b60) at mainproc.c:1359
#13 0x000000000041e710 in proc_encryption_packets (anchor=anchor@entry=0x0, a=a@entry=0x6b4b60) at mainproc.c:1264
#14 0x0000000000441495 in decrypt_message (filename=filename@entry=0x7fffffffe239 "/data/develop/local/eml.eml") at decrypt.c:89
#15 0x000000000040be04 in main (argc=1, argv=0x7fffffffdeb0) at gpg.c:3581
(gdb) c
Continuing.
gpg: Can't check signature: No public key

Breakpoint 1, log_get_errorcount (clear=clear@entry=0) at logging.c:77
77	    if( clear )
(gdb) bt
#0  log_get_errorcount (clear=clear@entry=0) at logging.c:77
#1  0x000000000040cff6 in g10_exit (rc=rc@entry=0) at gpg.c:4071
#2  0x000000000040a359 in main (argc=1, argv=0x7fffffffdeb0) at gpg.c:4044
(gdb) n
76	    int n = errorcount;
(gdb) n
77	    if( clear )
(gdb) p n
$1 = 1
(gdb) n
80	}
(gdb) 
g10_exit (rc=2, rc@entry=0) at gpg.c:4072
4072	  exit (rc);
(gdb) p rc
$2 = 2
(gdb) c
Continuing.
[Inferior 1 (process 32056) exited with code 02]


This log_error() is not called on evolution's message. As David said, this is more about a workaround on evolution's side, than on a fix, thus I'd rather move this to gnupg and ask them for their opinion (I can be wrong here, there can be a reason why it is valid to result in a decryption failure when the signature check failed).

More from the code of mainproc.c:

1967		write_status_text( STATUS_ERRSIG, buf );
1968		if( rc == G10ERR_NO_PUBKEY ) {
1969		    buf[16] = 0;
1970		    write_status_text( STATUS_NO_PUBKEY, buf );
1971		}
1972		if( rc != G10ERR_NOT_PROCESSED )
1973		    log_error(_("Can't check signature: %s\n"), g10_errstr(rc) );

when this happens, then the 'rc' is G10ERR_NO_PUBKEY, which might be considered a non-critical error. I believe that the line 1971 and line 1972 should be merged into:
1971		} else if( rc != G10ERR_NOT_PROCESSED )

It is really debatable whether the G10ERR_NO_PUBKEY is non-critical error or critical. What is the gpg2 command called that returns the error exit code?

(In reply to comment #9)
> It is really debatable whether the G10ERR_NO_PUBKEY is non-critical error or
> critical. What is the gpg2 command called that returns the error exit code?

gpg -d file_with_message_sent_by_tb+enigmail, as mentioned in #c5.

The gpg manual page says "-d .... If the decrypted file is  signed,  the  signature  is also  verified." So a problem with verification of a signature actually is a problem that should be reflected in the exit code.

(In reply to comment #11)
> So a problem with verification of a signature
> actually is a problem that should be reflected in the exit code.

See comment #6, the returned value should not be 2, but 1 instead, according to the same manual page.

That's really questionable as the 1 is reserved for signature errors. And this is not a signature error but a fatal error that's prevents gpg to verify the signature.

Could Evolution just display the decrypted message even on exit code 2 if there is anything outputted by gpg2?

I guess I can workaround it, but I thought more general fix will be better in gpg itself, to benefit for all applications.

If you think that "cannot verify" is not the same as "bad signature", then let's move this back to evolution-data-server,and I'll check for the workaround.

Cannot verify is definitely not the same as bad signature. However whether the 1 is strictly reserved for bad signature and not other problems with signature verification I don't know. I'd suppose this should be decided by upstream. Meanwhile you should probably solve this in the evolution-data-server.

Created upstream issue: https://bugs.g10code.com/gnupg/issue1500

Thanks for upstreaming it. I'm moving this back to eds, to find some workaround.

(In reply to comment #13)
> That's really questionable as the 1 is reserved for signature errors. And
> this is not a signature error but a fatal error that's prevents gpg to
> verify the signature.
> 

The weird thing is that the fatal error is gone once you add the key to the keyring! I'd expect that fatal errors shouldn't depend on such conditions - the signature should be reported as good (or bad) in all cases no matter what the keyring status is, only then should be reported trust level (key not known < key known but sender is not verified < verified key and sender).

For the record:
// sender key not included in the keyring
$ LC_ALL=C gpg -v -d message_from_enigmail+tb ; echo $?
Version: GnuPG v1.4.13 (GNU/Linux)
gpg: armor header: 
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
gpg: armor header: 
gpg: public key is B8318982
gpg: using subkey B8318982 instead of primary key 22C33E24

You need a passphrase to unlock the secret key for
user: "David Jaša <djasa>"
gpg: using subkey B8318982 instead of primary key 22C33E24
4096-bit RSA key, ID B8318982, created 2011-02-16 (main key ID 22C33E24)

gpg: public key is 1561DCD6
gpg: encrypted with RSA key, ID 1561DCD6
gpg: encrypted with 4096-bit RSA key, ID B8318982, created 2011-02-16
      "David Jaša <djasa>"
gpg: AES256 encrypted data
gpg: original file name=''
gpg: Signature made Mon May 13 08:36:50 2013 CEST using RSA key ID ...
gpg: Can't check signature: No public key
2

// sender key just added to the keyring


$ LC_ALL=C gpg -v -d message_from_enigmail+tb ; echo $?
Version: GnuPG v1.4.13 (GNU/Linux)
gpg: armor header: 
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
gpg: armor header: 
gpg: public key is B8318982
gpg: using subkey B8318982 instead of primary key 22C33E24

You need a passphrase to unlock the secret key for
user: "David Jaša <djasa>"
gpg: using subkey B8318982 instead of primary key 22C33E24
4096-bit RSA key, ID B8318982, created 2011-02-16 (main key ID 22C33E24)

gpg: public key is 1561DCD6
gpg: using subkey 1561DCD6 instead of primary key B15B82F5
gpg: encrypted with 2048-bit RSA key, ID 1561DCD6, created 2013-02-18
      "Neil Miao (SOC SysAdmin) <nmiao>"
gpg: encrypted with 4096-bit RSA key, ID B8318982, created 2011-02-16
      "David Jaša <djasa>"
gpg: AES256 encrypted data
gpg: original file name=''
gpg: Signature made Mon May 13 08:36:50 2013 CEST using RSA key ID ...
gpg: using PGP trust model
gpg: Good signature from "name <email>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 8C69 68D9 DBC5 3CB7 FF59  0437 98BA 74D6 B15B 82F5
gpg: textmode signature, digest algorithm SHA256
0

I committed this for 3.8.3+ and 3.9.2+ as (for the later):
https://git.gnome.org/browse/evolution-data-server/commit/?id=8541c66506f1c50e8d8d1cf1957832955579d346

and after looking into the code, it';s not much a workaround, because the code checked only for 0 return from gpg, thus if the signature would be considered invalid, then evolution would not show any data either.

I'd devel-acking this, patch available.

Given the upstream response:

> It has been told for ages that the value of the exit code is not a reliable way
> to get information from gpg.  Use the --status-fd information.

the patch should be IMO larger.

This is the what gpg writes to status file/fd on the message from enigmail without senders key in the keyring:

$ LC_ALL=C gpg --status-file /dev/stderr -d message_from_enigmail > /dev/null ; echo $?
[GNUPG:] ENC_TO 9568D668B8318982 1 0
[GNUPG:] USERID_HINT 9568D668B8318982 David Jaša <djasa>
[GNUPG:] NEED_PASSPHRASE 9568D668B8318982 B125CD0022C33E24 1 0

You need a passphrase to unlock the secret key for
user: "David Jaša <djasa>"
4096-bit RSA key, ID B8318982, created 2011-02-16 (main key ID 22C33E24)

[GNUPG:] GOOD_PASSPHRASE
[GNUPG:] ENC_TO <senders_subkey_id> 1 0
gpg: encrypted with RSA key, ID <senders_subkey_id>
[GNUPG:] NO_SECKEY <senders_subkey_id>
gpg: encrypted with 4096-bit RSA key, ID B8318982, created 2011-02-16
      "David Jaša <djasa>"
[GNUPG:] BEGIN_DECRYPTION
[GNUPG:] PLAINTEXT 74 1368427010 
gpg: Signature made Mon May 13 08:36:50 2013 CEST using RSA key ID <senders_key_id>
[GNUPG:] ERRSIG <senders_key_id> 1 8 01 1368427010 9
[GNUPG:] NO_PUBKEY <senders_key_id>
gpg: Can't check signature: No public key
[GNUPG:] DECRYPTION_OKAY
[GNUPG:] GOODMDC
[GNUPG:] END_DECRYPTION
2


and this with key in the keyring:

$ LC_ALL=C gpg --status-file /dev/stderr -d Plocha/message_from_enigmail > /dev/null ; echo $?
[GNUPG:] ENC_TO 9568D668B8318982 1 0
[GNUPG:] USERID_HINT 9568D668B8318982 David Jaša <djasa>
[GNUPG:] NEED_PASSPHRASE 9568D668B8318982 B125CD0022C33E24 1 0

You need a passphrase to unlock the secret key for
user: "David Jaša <djasa>"
4096-bit RSA key, ID B8318982, created 2011-02-16 (main key ID 22C33E24)

[GNUPG:] GOOD_PASSPHRASE
[GNUPG:] ENC_TO <sender_subkey_id> 1 0
gpg: encrypted with 2048-bit RSA key, ID <sender_subkey_id>, created 2013-02-18
      "sender <sender@email>"
[GNUPG:] NO_SECKEY <sender_subkey_id>
gpg: encrypted with 4096-bit RSA key, ID B8318982, created 2011-02-16
      "David Jaša <djasa>"
[GNUPG:] BEGIN_DECRYPTION
[GNUPG:] PLAINTEXT 74 1368427010 
gpg: Signature made Mon May 13 08:36:50 2013 CEST using RSA key ID B15B82F5
[GNUPG:] SIG_ID ObZapsVl9Jv1MYmb+u4J8GkOlpQ 2013-05-13 1368427010
[GNUPG:] GOODSIG <sender_key_id> sender_name <sender@email>
gpg: Good signature from "sender <sender@email>"
[GNUPG:] VALIDSIG <sender_key_id> 2013-05-13 1368427010 0 4 0 1 8 01 <sender_key_id>
[GNUPG:] TRUST_UNDEFINED
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: <sender_key_fingerprint>
[GNUPG:] DECRYPTION_OKAY
[GNUPG:] GOODMDC
[GNUPG:] END_DECRYPTION
0


So the information from status-{file,fd} seems to be reliable.

(In reply to comment #19)
> Given the upstream response:
> 
> > It has been told for ages that the value of the exit code is not a reliable way
> > to get information from gpg.  Use the --status-fd information.
> 
> the patch should be IMO larger.

The attached patch does basically it: if there was anything decrypted, then it ignores returned value from gpg and drives all the rest behaviour based on the status data. Patch size doesn't tell anything about its functionality. :)

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-1540.html

Removing needinfo on closed bugs.