Bug 1385720 - mysql server crashed after VM restart
Summary: mysql server crashed after VM restart
Keywords:
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: mysql
Version: 6.8
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Jakub Dorňák
QA Contact: qe-baseos-daemons
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-10-17 15:00 UTC by Tomas Pelka
Modified: 2016-10-25 12:25 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-10-25 12:25:08 UTC
Target Upstream Version:


Attachments (Terms of Use)
my.cnf (304 bytes, text/plain)
2016-10-17 16:42 UTC, Tomas Pelka
no flags Details

Description Tomas Pelka 2016-10-17 15:00:38 UTC
Description of problem:
61017 09:52:18 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
161017  9:52:18  InnoDB: Initializing buffer pool, size = 8.0M
161017  9:52:18  InnoDB: Completed initialization of buffer pool
InnoDB: Log scan progressed past the checkpoint lsn 13 1572000176
161017  9:52:18  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 13 1572005480
161017  9:52:18  InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 13:52:18 UTC - mysqld got signal 11 ;
53 This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
54 We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.

key_buffer_size=8384512
read_buffer_size=131072
max_used_connections=0
max_threads=300
55 thread_count=0
connection_count=0
56 It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 664112 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
57 58 59 60 61 62 63 64 65 66 67 stack_bottom = 0 thread_stack 0x40000
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 /usr/libexec/mysqld(my_print_stacktrace+0x29) [0x8509f9]
/usr/libexec/mysqld(handle_fatal_signal+0x483) [0x6a3f13]
/lib64/libpthread.so.0() [0x3532a0f7e0]
/usr/libexec/mysqld(page_cur_insert_rec_low+0x288) [0x7999b8]
/usr/libexec/mysqld(page_cur_parse_insert_rec+0x4eb) [0x79a55b]
/usr/libexec/mysqld() [0x787a0d]
/usr/libexec/mysqld(recv_recover_page+0x34f) [0x7894cf]
/usr/libexec/mysqld(buf_page_io_complete+0x548) [0x74a888]
/usr/libexec/mysqld(fil_aio_wait+0xfa) [0x762a3a]
/usr/libexec/mysqld() [0x7c8ab0]
/lib64/libpthread.so.0() [0x3532a07aa1]
/lib64/libc.so.6(clone+0x6d) [0x35326e8aad]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
161017 09:52:18 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
161017 10:52:25 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
161017 10:52:26  InnoDB: Initializing buffer pool, size = 8.0M
161017 10:52:26  InnoDB: Completed initialization of buffer pool
InnoDB: Log scan progressed past the checkpoint lsn 13 1572000176
161017 10:52:26  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 13 1572005480
161017 10:52:26  InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 14:52:26 UTC - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.

58 key_buffer_size=8384512
read_buffer_size=131072
max_used_connections=0
59 max_threads=300
thread_count=0
connection_count=0
60 It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 664112 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0
61 Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
62 63 64 65 66 67 68 69 70 stack_bottom = 0 thread_stack 0x40000
71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 /usr/libexec/mysqld(my_print_stacktrace+0x29) [0x8509f9]
/usr/libexec/mysqld(handle_fatal_signal+0x483) [0x6a3f13]
/lib64/libpthread.so.0() [0x3532a0f7e0]
/usr/libexec/mysqld(page_cur_insert_rec_low+0x288) [0x7999b8]
/usr/libexec/mysqld(page_cur_parse_insert_rec+0x4eb) [0x79a55b]
/usr/libexec/mysqld() [0x787a0d]
/usr/libexec/mysqld(recv_recover_page+0x34f) [0x7894cf]
/usr/libexec/mysqld(buf_page_io_complete+0x548) [0x74a888]
/usr/libexec/mysqld(fil_aio_wait+0xfa) [0x762a3a]
/usr/libexec/mysqld() [0x7c8ab0]
/lib64/libpthread.so.0() [0x3532a07aa1]
/lib64/libc.so.6(clone+0x6d) [0x35326e8aad]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
161017 10:52:26 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended

Version-Release number of selected component (if applicable):
mysql-5.1.73-7.el6

How reproducible:
100% for me

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
I would like to get data from there if possible.

Comment 3 Tomas Pelka 2016-10-17 16:42:55 UTC
Created attachment 1211473 [details]
my.cnf

Comment 4 Honza Horak 2016-10-25 12:25:08 UTC
Tomas, we talked together offline, which didn't help much. Anyway, as for this bug, I think we cannot do much, unless we will have a reproducer that would "break" the data directory from working state to broken state, the same way as it happened to you. Closing this bug for now, but feel free to re-open if there are some more information.


Note You need to log in before you can comment on or make changes to this bug.