Bug 1048293 - RFE/TECH DEBT: support mysql 5.6 or mariadb 10.x database for Zanata
Summary: RFE/TECH DEBT: support mysql 5.6 or mariadb 10.x database for Zanata
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Zanata
Classification: Retired
Component: DatabaseChange
Version: 3.0
Hardware: x86_64
OS: Linux
medium
high
Target Milestone: ---
: 3.7
Assignee: Sean Flanigan
QA Contact: Zanata-QA Mailling List
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-01-03 15:57 UTC by Dmitry
Modified: 2015-07-22 02:19 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: 1
Clone Of:
Environment:
Last Closed: 2015-07-22 02:19:55 UTC
Embargoed:


Attachments (Terms of Use)
server log (122.19 KB, text/plain)
2014-01-03 16:02 UTC, Dmitry
no flags Details
server.log (33.77 KB, text/plain)
2015-03-30 01:24 UTC, Ding-Yi Chen
no flags Details
server..log after rebuild db (49.51 KB, text/plain)
2015-03-30 01:46 UTC, Ding-Yi Chen
no flags Details

Description Dmitry 2014-01-03 15:57:39 UTC
User-Agent:       Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/31.0.1650.63 Chrome/31.0.1650.63 Safari/537.36
Build Identifier: 

I tried to start the same instance of Zanata on two different versions of PostgreSQL. On one version of PostgreSQL (older version) it started without any issue on other version (more recent) it fails.

With this PostgreSQL (older version -- 5.5.34) Zanata server works fine:

mysql  Ver 14.14 Distrib 5.5.34, for debian-linux-gnu (x86_64) using readline 6.2

Connection id:		59
Current database:	zanata
Current user:		zanata@localhost
SSL:			Not in use
Current pager:		stdout
Using outfile:		''
Using delimiter:	;
Server version:		5.5.34-0ubuntu0.12.10.1 (Ubuntu)
Protocol version:	10
Connection:		Localhost via UNIX socket
Server characterset:	latin1
Db     characterset:	latin1
Client characterset:	utf8
Conn.  characterset:	utf8
UNIX socket:		/var/run/mysqld/mysqld.sock
Uptime:			4 hours 21 min 34 sec


With this PostgreSQL (newer version -- 5.6.12) Zanata server fails at start:

mysql  Ver 14.14 Distrib 5.5.34, for debian-linux-gnu (x86_64) using readline 6.2

Connection id:		76
Current database:	zanata
Current user:		zanata.3.121
SSL:			Not in use
Current pager:		stdout
Using outfile:		''
Using delimiter:	;
Server version:		5.6.12 Source distribution
Protocol version:	10
Connection:		cambiatica.kiev.zoral.com.ua via TCP/IP
Server characterset:	latin1
Db     characterset:	latin1
Client characterset:	utf8
Conn.  characterset:	utf8
TCP port:		23306
Uptime:			6 hours 38 min 38 sec

Logfile with error see in the attachment.



Reproducible: Always

Steps to Reproduce:
1. Configure Zanata datasource to use PostgreSQL 5.6.12
2. Start Zanata server.
3. Catch exceptions on the console.



OS:
    $ uname -a
    Linux titan 3.5.0-45-generic #68-Ubuntu SMP Mon Dec 2 21:58:52 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

Version of Zanata server: 3.0.2

Java version: 
    $ java -version
    java version "1.7.0_40"
    Java(TM) SE Runtime Environment (build 1.7.0_40-b43)
    Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode)

Comment 1 Dmitry 2014-01-03 16:02:26 UTC
Created attachment 845002 [details]
server log

Comment 2 Sean Flanigan 2014-01-06 01:32:36 UTC
You mean MySQL, not PostgreSQL, right?  Unfortunately, Zanata doesn't yet support  MySQL 5.6, let alone PostgreSQL.

It seems that changing a column to be autoincrement violates foreign key constraints starting with MySQL 5.6, so we'll need to modify changeset db/changelogs/db.changelog-1.5.xml::8::aeng to drop the constraint and recreate it.  There could be quite a few other changes required too.

By the way, Dmitry, it looks like your databases are using latin1 for 'Db characterset'.  All of our testing is with UTF-8 databases, so you may have trouble storing non-ASCII text in Zanata (even non-ASCII Latin-1 text).

Comment 5 Sean Flanigan 2014-01-06 06:30:59 UTC
I think this shows what we need to do in that changeset: http://stackoverflow.com/a/13606801/14379

Comment 6 Dmitry 2014-01-06 07:24:04 UTC
(In reply to Sean Flanigan from comment #2)
> You mean MySQL, not PostgreSQL, right?

Yes, you are right. I meant MySql.

> By the way, Dmitry, it looks like your databases are using latin1 for 'Db
> characterset'.  All of our testing is with UTF-8 databases, so you may have
> trouble storing non-ASCII text in Zanata (even non-ASCII Latin-1 text).

I thought that option "characterEncoding=UTF-8" in the JDBC connection string should negotiate correct sort order. BTW in the documentation at

https://github.com/zanata/zanata-server/wiki/Database-Setup#Setup_for_Zanata

I found this statement to create database:

CREATE DATABASE zanata /*!40100 DEFAULT CHARACTER SET utf8 */;

which I used to. If it so important it should be replaced on more precise version:

CREATE DATABASE zanata CHARACTER SET utf8 COLLATE utf8_general_ci;

Comment 8 Sean Flanigan 2014-08-15 01:54:14 UTC
Fedora 21 isn't far away, and it doesn't include mysql 5.5, only mysql 5.6 and mariadb 10.0.  We need to do something about this.

http://fedoraproject.org/wiki/Releases/21/Schedule

Comment 9 Sean Flanigan 2015-01-14 07:39:43 UTC
Zanata 3.6 seems to work with MariaDB 10.0 on some of our Fedora 21 dev boxes, although we haven't given it a great deal of testing: our main test/prod servers still use RHEL 6 with MySQL 5.1.  MariaDB 10 seems to be more compatible with MySQL 5.5 than MySQL 5.6 is, at least for Zanata's purposes.  

Zanata should be working with MySQL 5.1 up to 5.5, or MariaDB 5.1 or later (at least up to 10.0).  On Fedora 21+, please install the package 'mariadb-server', not 'community-mysql-server'.  I think you need 'mariadb-server' on Ubuntu as well.

We are not currently planning any development work to target MySQL 5.6 or later.

Comment 10 Sean Flanigan 2015-01-14 07:42:27 UTC
Carlos, Patrick, one thing though: our functional tests use com.jcabi:jcabi-mysql-maven-plugin, which may be able to work with mariadb, but doesn't make it as easy to install.  Right now these tests use mysql 5.5, but at some point we may need to convince jcabi-mysql-maven-plugin to install mariadb.

Comment 11 Sean Flanigan 2015-03-30 01:17:49 UTC
Reopening.  According to dchen, this has been happening on MariaDB too.  So it looks like Zanata won't run on Fedora 21+ either with MySQL or MariaDB.


A similar problem was reported here:
  https://groups.google.com/a/openmrs.org/d/msg/dev/lX5Bo__O9eM/1A46a5NJvEwJ

and fixed here:
  https://github.com/openmrs/openmrs-core/pull/622/files

(Compare liquibase changeset 1-fix:sunbiz with 3-increase-privilege-col-size-person_attribute_type:dkayiw.  We basically have to drop the constraint, change the column and create the constraint again.)

Comment 12 Ding-Yi Chen 2015-03-30 01:24:56 UTC
Created attachment 1008135 [details]
server.log

Run /usr/share/wildfly/bin/standalone.sh
failed with 

Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Can't DROP 'sourceRef'; check that column/key exists                                              

Full server.log is attached

=======

rpm -q wildfly
wildfly-8.1.0-3.fc21.noarch

rpm -qa mariadb\*

mariadb-10.0.15-4.fc21.x86_64
mariadb-config-10.0.15-4.fc21.x86_64
mariadb-common-10.0.15-4.fc21.x86_64
mariadb-libs-10.0.15-4.fc21.x86_64
mariadb-errmsg-10.0.15-4.fc21.x86_64
mariadb-server-10.0.15-4.fc21.x86_64

Comment 13 Ding-Yi Chen 2015-03-30 01:46:19 UTC
Created attachment 1008147 [details]
server..log after rebuild db

Rebuild db with following command

echo 'drop database if exists zanata; create database zanata /*!40100 DEFAULT CHARACTER SET utf8 */;' |  mysql -u root

Then run 

/usr/share/wildfly/bin/standalone.sh

Full server.log is attached.

Comment 14 Sean Flanigan 2015-04-01 05:09:50 UTC
Pull request: https://github.com/zanata/zanata-server/pull/756


Note You need to log in before you can comment on or make changes to this bug.