Bug 1970502

An unexpected error occurred while creating the scratch build and has been automatically reported. Sorry!

This one is still a BETA release, so I would wait until it's in the STABLE state.

Latest upstream release: 3.2.3
Current version/release in rawhide: 3.1.13-1.fc35
URL: https://mariadb.com/kb/en/library/about-mariadb-connector-c/

Please consult the package updates policy before you issue an update to a stable branch: https://docs.fedoraproject.org/en-US/fesco/Updates_Policy/


More information about the service that created this bug can be found at: https://fedoraproject.org/wiki/Upstream_release_monitoring


Please keep in mind that with any upstream change, there may also be packaging changes that need to be made. Specifically, please remember that it is your responsibility to review the new version to ensure that the licensing is still correct and that no non-free or legally problematic items have been added upstream.


Based on the information from anitya: https://release-monitoring.org/project/16939/

An unexpected error occurred while creating the scratch build and has been automatically reported. Sorry!

The PR is ready here: https://src.fedoraproject.org/rpms/mariadb-connector-c/pull-request/12

The update is currently under review

I'd like to see a test to be created for the change of the default charset.
  https://mariadb.com/kb/en/mariadb-connector-c-320-release-notes/#notable-changes
  |  Default character set is now utf8mb4
  |  Character set utf8 will be mapped to utf8mb3

There are several components that might exists in several versions:
 * MariaDB connector C / C++
   3.1 release vs >= 3.2 releases
 * MariaDB connector ODBC
   - builds on top of MariaDB connector C
   - need to verify if affected at all
   - need to find out which was the first release built on top of the new Connector C
 * MariaDB server
   - release 10.3, 10.5, 10.6
   - need to find out which was the first release built on top of the new Connector C (if any)
And three main charsets that needs to be tested
 * utf8, utf8mb3
 * utf8mb4
 * latin1

The test should:
 - in a description list the versions that are affected and describe how they are affected.
 - test the behaviour of all the versions when the charset _is set_ by the user or application developer
 - test the behaviour when the charset _is NOT set_ by the user or application developer
   - that _might_ cause issues for applications built in the past, running on the latest versions of MariaDB software stack

We should also check if the users / app developers need to do anything about following upstream change:
  |  Character set utf8 will be mapped to utf8mb3

Tests are for now available at: https://github.com/zmiklank/mariadb_charset_test
Results of the tests are as follows:

1. testing explicitly set utf8mb4 and latin1 charsets in all possible combinations of connection charset and database charset.
   Incorrect combinations are as follows:

    1. write with connection charset: latin1, database charset: utf8mb4
       - read with connection charset utf8mb4 -> characters that are in the utf8mb4 and not in the latin1 charset printed incorrectly

    2. write with connection charset: utf8mb4, database charset: latin1
       - cannot write to db chars, which are in utf8mb4, but not in latin1

    3. write with connection charset: latin1, database charset: latin1
       - read with connection charset utf8mb4 -> characters that are in the utf8mb4 and not in the latin1 charset printed incorrectly

    4. write with connection charset: utf8mb4, database charset: utf8mb4
       - read with connection charset latin1 -> characters that are in the utf8mb4 and not in the latin1 charset printed incorrectly

2. testing default connection and default database charset (not explicitly changed by the user).
   Testing variations in versions mariadb 10.5, 10.6 - since connector-c 3.2 is firstly embedded in mariadb 10.6, and mariadb-connector-c, versions 3.1, 3.2:

   1.  mariadb-connector-c  3.1.13-1.fc34:q
       mariadb-server       10.5.12-1
       with result: success, as expected

   2.  mariadb-connector-c  3.2.3-1.fc34 
       mariadb-server       10.5.12-3
       with result: cannot write to db chars, which are in the utf8mb4 charset, but not in the latin1 charset

   3.  mariadb-connector-c  3.2.3-1.fc34 
       mariadb-server       10.6.4.-1
       with result: cannot write to db chars, which are in the utf8mb4 charset, but not in the latin1 charset

What remains is testing the mariadb-connector-odbc.

mariadb-connector-odbc test results:

currently is the newest upstream mariadb-connector-odbc compiled with the mariadb-connector-c v. 3.1.13 (latin1 default charset)

Writing and reading from a db via the odbc connector was tested with default character sets (not explicitly set by the user) in following combinations of versions:
   1.  mariadb-connector-odbc 3.1.13 
       mariadb-connector-c    3.1.13-1.fc34
       mariadb-server         10.5.12-1
       with result: write failed with error: "[ISQL]ERROR: Could not SQLExecute" for entries containing the utf8mb4 chars
                    read did not failed, but the uf8mb4 based entries were not present

   2.  mariadb-connector-odbc 3.1.13 
       mariadb-connector-c    3.2.3-1.fc34 
       mariadb-server         10.5.12-3
       with result: write failed with error: "[ISQL]ERROR: Could not SQLExecute" for entries containing the utf8mb4 chars
                    read did not failed, but the uf8mb4 based entries were not present

   3.  mariadb-connector-odbc 3.1.13 
       mariadb-connector-c    3.2.3-1.fc34 
       mariadb-server         10.6.4.-1
       with result: write failed with error: "[ISQL]ERROR: Could not SQLExecute" for entries containing the utf8mb4 chars
                    read did not failed, but the uf8mb4 based entries were not present

Reason is, that the odbc connector already used the utf8mb4 character set for unicode connections as default one (but mariadb's default is latin1),
as can be seen, e.g., in the ma_connection.s file:

  /* If a client character set was specified in DSN, we will always use it.
     Otherwise for ANSI applications we will use the current character set,
     for unicode connections we use utf8
  */

Thank you Zuzka for all of the testings, great job.

So if I understand it correctly, if we would ship mariadb-connector-c-3.2.3 with mariadb-10.5/10.6 it wouldn't work correctly by default right?

Also, the mariadb-connector-odbc is not working with any mariadb-server and mariadb-connector-c we have by default, when using the utf8mb4 characters right?

If the above assumptions are correct, we should maybe discuss moving the mariadb-server default charset to the 'utf8mb4' with upstream.

> So if I understand it correctly, if we would ship mariadb-connector-c-3.2.3 with mariadb-10.5/10.6 it wouldn't work correctly by default right?

Yes, for characters that are in unicode, but not in latin1 charset.
However, problem could rise from changing charset of the existing database [1], I guess.

> Also, the mariadb-connector-odbc is not working with any mariadb-server and mariadb-connector-c we have by default, when using the utf8mb4 characters right?

Yes, in general, there is no possibility to encode all unicode characters (up to 4B/char) to 1B/char latin1 encoding, so the characters which are in unicode,
but not in latin1 can not be written by the odbc connector with default settings to the mariadb database with default settings.
However, the odbc connector connection settings can be set in its DSN entries.



[1] https://mariadb.com/kb/en/alter-database/

Rebased to the 3.2.5 (newest) release in the PR comment#5

Closing in favor of https://bugzilla.redhat.com/show_bug.cgi?id=2055613

*** This bug has been marked as a duplicate of bug 2055613 ***