Bug 1970829

Summary: [MariaDB 10.5] .mysql_history shows \040 instead of spaces
Product: Red Hat Enterprise Linux 8 Reporter: Jordi Sanfeliu <jordi>
Component: mariadbAssignee: Michal Schorm <mschorm>
Status: CLOSED NOTABUG QA Contact: RHEL CS Apps Subsystem QE <rhel-cs-apps-subsystem-qe>
Severity: unspecified Docs Contact: Lenka Špačková <lkuprova>
Priority: unspecified    
Version: ---CC: carl, databases-maint, hhorak, ljavorsk, lkuprova, pkubat, tis
Target Milestone: beta   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
MariaDB 10.4 and later uses the libedit implementation of the underlying software managing the MariaDB command history (the .mysql_history file) instead of the previously used readline library. This change impacts users working directly with the .mysql_history file. Note that .mysql_history is a file managed by the MariaDB or MySQL applications, and users should not work with the file directly. The human-readable appearance is coincidental. NOTE To increase security, you can consider not maintaining a history file. To disable the command history recording: 1. Remove the .mysql_history file if it exists. 2. Use either of the following approaches: * Set the MYSQL_HISTFILE variable to /dev/null and include this setting in any of your shell’s startup files. * Change the .mysql_history file to a symbolic link to /dev/null: $ ln -s /dev/null $HOME/.mysql_history
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-07-12 15:28:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jordi Sanfeliu 2021-06-11 09:57:04 UTC
Description of problem:

The lines in the file .mysql_history' are saved showing \040 instead of spaces, which makes it difficult to read it or copy it.

Version-Release number of selected component (if applicable):


How reproducible:

Always


Steps to Reproduce:
1. use the command 'mariadb' to connect to database.
2. enter any command that includes spaces and then quit.
3. cat the newly created file '.mysql_history'.

# mariadb -u root -p
Enter password: 
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 15
Server version: 10.5.9-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.001 sec)

MariaDB [(none)]> ^DBye
[root@wapp ~]# 
[root@wapp ~]# cat .mysql_history 
_HiStOrY_V2_
FLUSH\040PRIVILEGES;


Actual results:

The file '.mysql_history' shows \040 instead of a space.


Expected results:

Lines should be the command exactly as it was entered.


Additional info:

MariaDB 10.3 don't has this issue.

Comment 1 Michal Schorm 2021-07-06 17:25:20 UTC
I tested this issue on several machines, OSs, MariaDB versions and builds combinations.

It turns out, it is reproducible on the combination you reported:
  RHEL 8.4    mariadb-10.5.9-1.module+el8.4.0+10457+88cd7ba1.x86_64
However also in Fedora for
  Fedora 34   mariadb-10.5.9-5.fc34.x86_64
  Fedora 34   mariadb-10.5.11-1.fc34.x86_64
  Fedora 33   mariadb-10.4.19-1.fc33.x86_64
And also with the upstream RPMs:
  Fedora 34   MariaDB-client-10.5.11-1.fc34.x86_64
  Fedora 34   MariaDB-client-10.4.20-1.fc33.x86_64

The issue comes with upgrade between MariaDB 10.3 and 10.4.
However it doesn't seem like a MariaDB issue. This behaviour seems to be intended, my MySQL upstream, where the same change appeared between 5.5.38 & 5.6.18.

The format of the ".mysql_history" file changed from version 1 to version 2; that's what the intitial line with "_HiStOrY_V2_" means.

The new format also required a background change; switch from a 'readline' to 'libedit' underlying software.

This new behaviour is reported as an issue on the MySQL upstream, you can see more here:
  https://bugs.mysql.com/bug.php?id=72108
  https://bugs.mysql.com/bug.php?id=69991
  https://bugs.mysql.com/bug.php?id=68925
  http://haxx.sinequanon.net/2012/01/mysql-history-munging/

---

You can verify the underlying implementation by running "mysql --version":

  # mysql --version
  mysql  Ver 15.1 Distrib 10.3.27-MariaDB, for Linux (x86_64) using readline 5.1

  # mysql --version
  mysql  Ver 15.1 Distrib 10.5.11-MariaDB, for Linux (x86_64) using  EditLine wrapper

---

There are more peculiarities to the libedit builds.
For example multi-line statements are logged twice. First there is a record of each line. Then there is a record of the statement as a whole.

---

This is not something we will change.
We want our packages to be close to the upstream ones, as we believe this brings a great number of benefits to all of the users, developers and maintainers of all of the similar packages.

Please look at the ".mysql_history" file as file with a binary format, managed by the MariaDB or MySQL application, and with which the users are not expected to work with directly. The fact it appears (atleast partly) human-readable, is just incidental, thanks to the underlying implementation of the code that manages the history.

---

I'll insist to mention this in our documentation regarding MariaDB 10.5

Comment 4 Jordi Sanfeliu 2021-07-09 06:58:59 UTC
(In reply to Michal Schorm from comment #1)

> This is not something we will change.
> We want our packages to be close to the upstream ones, as we believe this
> brings a great number of benefits to all of the users, developers and
> maintainers of all of the similar packages.

I understand, reverting to readline could have undefined new results.


> Please look at the ".mysql_history" file as file with a binary format,
> managed by the MariaDB or MySQL application, and with which the users are
> not expected to work with directly. The fact it appears (atleast partly)
> human-readable, is just incidental, thanks to the underlying implementation
> of the code that manages the history.

It's a pity that, in a slowly way, some software is abandoning the human-readable text/plain files.


> I'll insist to mention this in our documentation regarding MariaDB 10.5

Thanks for your assistance.

Comment 6 Michal Schorm 2021-07-12 15:28:18 UTC
Thank you Lenka,

I'm now closing this ticket as a "Not a bug" since this is the intended behaviour of the software.

Thank you for reaching us.