Bug 1051924

When the store recovery process encounters a journal where the last logical file is full and the last record in that file is also incomplete (ie this record would ordinarily span this journal file and the next if it were present), then the recovery process crashes with a segfault.

There is no specific reproducer, but using large message sizes in a test will increase the probability that this error will occur, and killing the broker suddenly (ie with signal SIGKILL) increases the probability that this error will occur.

Although untested, this error could also be synthesized by deleting the last file(s) in a valid journal provided that the last record in the remaining journal files is still undequeued and spans to a deleted file. ie Enqueue enough large records to fill 2 or 3 journal files. Recover the journal using info level logging, and look at the table of recovered files and their logical ordering. Starting at the second-last file, look for a journal file using hexdump or a journal analysis tool where the last record spans that journal file and the next. Then delete the following journal file(s).