Bug 2084161

> I need support for PKCS#7 padding in gnutls_cipher_*() functions to implement this correctly.

I have been thinking about how to integrate this feature into the existing API. If we pad/unpad transparently, we would need a complete new API, not only new cipher IDs or a variant with a flag to indicate padding, because the decrypt function needs to report the actual length after unpadding, something like:

  int gnutls_cipher_decrypt3(gnutls_cipher_algorithm_t cipher, const uint8_t *ciphertext, size_t ciphertext_len, uint8_t *plaintext, size_t *plaintext_len);

Note the last argument is of type `size_t *`. The encrypt function could have the same signature as gnutls_cipher_encrypt2, though it requires a check whether ciphertext has enough room to hold the last block (if we want to avoid extra allocation). In any case implementing this approach wouldn't be straightforward.

The alternative approach would be to just provide low-level functions that pad/unpad a single block:

  int gnutls_pkcs7_pad(uint8_t *block, size_t block_len, size_t plaintext_len);
  int gnutls_pkcs7_unpad(const uint8_t *block, size_t block_len, size_t *plaintext_len);

Do you think if this is acceptable?

I've ended up with the following API:

```c
/**
 * gnutls_cipher_encrypt3:
 * @handle: is a #gnutls_cipher_hd_t type
 * @ptext: the data to encrypt
 * @ptext_len: the length of data to encrypt
 * @ctext: the encrypted data
 * @ctext_len: the length of encrypted data (initially must hold the maximum available size)
 * @flags: flags for padding
 *
 * This function will encrypt the given data using the algorithm
 * specified by the context. For block ciphers, if the @ptext_len is
 * not a multiple of the block size, the last block is padded
 * according to @flags, typically %GNUTLS_CIPHER_PADDING_PKCS7. In
 * that case, @ctext must hold enough space to store padded cipher
 * text and @ctext_len is updated to be a multiple of the block
 * size. The initial size can be obtained by calling this function
 * with @ctext set to %NULL.
 *
 * Returns: Zero or a negative error code on error.
 *
 * Since: 3.7.7
 **/
int
gnutls_cipher_encrypt3(gnutls_cipher_hd_t handle,
		       const void *ptext, size_t ptext_len,
		       void *ctext, size_t *ctext_len,
		       unsigned flags);

/**
 * gnutls_cipher_decrypt3:
 * @handle: is a #gnutls_cipher_hd_t type
 * @ctext: the data to decrypt
 * @ctext_len: the length of data to decrypt
 * @ptext: the decrypted data
 * @ptext_len: the available length for decrypted data
 * @flags: flags for padding
 *
 * This function will decrypt the given data using the algorithm
 * specified by the context. If @flags is specified, padding for the
 * decrypted data will be removed accordingly and @ptext_len will be
 * updated.
 *
 * Returns: Zero or a negative error code on error.
 *
 * Since: 3.7.7
 **/
int
gnutls_cipher_decrypt3(gnutls_cipher_hd_t handle,
		       const void *ctext, size_t ctext_len,
		       void *ptext, size_t *ptext_len,
		       unsigned flags);
```

See https://gitlab.com/gnutls/gnutls/-/merge_requests/1611 for details.

The API looks fine for me. It allows to add more flags in future. I've commented the upstream commit.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (gnutls bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:2522