Bug 646861 - Massive data loss after update
Summary: Massive data loss after update
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora EPEL
Classification: Fedora
Component: rabbitmq-server
Version: el5
Hardware: Unspecified
OS: Unspecified
low
high
Target Milestone: ---
Assignee: Hubert Plociniczak
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-10-26 13:48 UTC by Sven Lankes
Modified: 2011-04-14 01:13 UTC (History)
3 users (show)

Fixed In Version: rabbitmq-server-2.2.0-1.el5
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-02-07 17:53:52 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Sven Lankes 2010-10-26 13:48:58 UTC
On updating a el5 machine yesterday, I got the rabbitmq-server update from 2.0 to 2.1

This resulted in:

1. rabbitmq-server shutting down during update - as far as I know this is a no-go?
2. Total data loss. All vhosts and users were gone after the update.

Comment 1 Hubert Plociniczak 2010-10-26 16:55:03 UTC
(In reply to comment #0)
> 1. rabbitmq-server shutting down during update - as far as I know this is a
> no-go?

This is just how the packaging scripts work. Also you cannot do it with current "non-packaged" version either. This is not a bug in the package rather an enhancement request in the application itself. I would suggest asking on the rabbitmq mailing list first.

> 2. Total data loss. All vhosts and users were gone after the update.
Release notes for 2.1.0:
"Upgrading
=========
The database schema has changed since the last release (2.0.0). When
starting, the RabbitMQ server will detect the existence of an old
database and will move it to a backup location, before creating a
fresh, empty database, and will log a warning. If your RabbitMQ
installation contains important data then we recommend you contact
support for assistance with the upgrade.
"

Comment 2 Sven Lankes 2010-10-26 22:11:56 UTC
>> 1. rabbitmq-server shutting down during update - as far as I know this is a
>> no-go?

> This is just how the packaging scripts work. Also you cannot do it with
> current "non-packaged" version either. This is not a bug in the package 
> rather an enhancement request in the application itself. I would suggest 
> asking on the rabbitmq mailing list first.

I think you're missing my point.

Let's say I have working rabbitmq server.

After updating it to a new release I do no longer have a working rabbitmq 
server because the update has stopped the server and not restarted it.

That is clearly a packaging issue. 
 
> > 2. Total data loss. All vhosts and users were gone after the update.
> Release notes for 2.1.0:
> "Upgrading
> =========
> The database schema has changed since the last release (2.0.0). When
> starting, the RabbitMQ server will detect the existence of an old
> database and will move it to a backup location, before creating a
> fresh, empty database, and will log a warning. If your RabbitMQ
> installation contains important data then we recommend you contact
> support for assistance with the upgrade.
> "

So you're suggesting I read the release notes for each and every package 
update I'm installing in the future?

An update like this is clearly not acceptable for a stable EL - see also:

http://fedoraproject.org/wiki/EPEL/GuidelinesAndPolicies

Comment 3 Peter Lemenkov 2010-10-27 09:43:27 UTC
Folks, calm down. Let's think how we could resolve this issue w/o additional damage and with mutual respect.

I personally welcome ver. 2.1.0  (and newly released 2.1.1) in EL-5 since it offers several bugfixes over 2.0.x. However I'm really sad because of changed DB scheme.

Hubert, are there are any means to do automatic conversion from old DB to new on?

Comment 4 Hubert Plociniczak 2010-10-27 10:02:14 UTC
(In reply to comment #2)
> >> 1. rabbitmq-server shutting down during update - as far as I know this is a
> >> no-go?
> 
> > This is just how the packaging scripts work. Also you cannot do it with
> > current "non-packaged" version either. This is not a bug in the package 
> > rather an enhancement request in the application itself. I would suggest 
> > asking on the rabbitmq mailing list first.
> 
> I think you're missing my point.
> 
> Let's say I have working rabbitmq server.
> 
> After updating it to a new release I do no longer have a working rabbitmq 
> server because the update has stopped the server and not restarted it.
> 
> That is clearly a packaging issue. 

Ok, I thought you were talking about the hot code swapping since we are in the Erlang world.
But anyway, stopping the server during the upgrade is nothing unusual just look at the specs of other erlang packages like ejabbberd or couchdb so rabbitmq-server is following the standard in this case.

> 
> So you're suggesting I read the release notes for each and every package 
> update I'm installing in the future?
> 
> An update like this is clearly not acceptable for a stable EL - see also:
> 
> http://fedoraproject.org/wiki/EPEL/GuidelinesAndPolicies

This was explanation for your problem, and the data was not lost it was backup-ed up. 

> I personally welcome ver. 2.1.0  (and newly released 2.1.1) in EL-5 since it
> offers several bugfixes over 2.0.x. However I'm really sad because of changed
> DB scheme.

As far as I remember the db format has changed between many of the releases and if we were to stay without the db scheme then many of the critical bugs wouldn't be visible in the stable EL-5 branch, which I think is less evil. 

> Hubert, are there are any means to do automatic conversion from old DB to new
> on?

Nothing I am aware of but I will talk to the rabbit team.

Comment 5 Rex Dieter 2010-10-27 16:27:04 UTC
Not that it helps a lot now this is in updates, but for posterity, this kind of change runs counter to both the letter and intent of EPEL update policy, per
http://fedoraproject.org/wiki/EPEL/GuidelinesAndPolicies#A_major_version_update

Comment 6 Hubert Plociniczak 2010-11-01 19:59:22 UTC
(In reply to comment #4)
> > I personally welcome ver. 2.1.0  (and newly released 2.1.1) in EL-5 since it
> > offers several bugfixes over 2.0.x. However I'm really sad because of changed
> > DB scheme.
> 
> As far as I remember the db format has changed between many of the releases and
> if we were to stay without the db scheme then many of the critical bugs
> wouldn't be visible in the stable EL-5 branch, which I think is less evil. 
> 
> > Hubert, are there are any means to do automatic conversion from old DB to new
> > on?
> 
> Nothing I am aware of but I will talk to the rabbit team.

I talked to the rabbit team and fortunately they are actively working on that problem. So there are really high chances that 2.0.x -> 2.1.0 db schema change was the last one that affects the end users and in the next release the actual db migration will be automatic. I will leave the bug open until then.

Comment 7 Fedora Update System 2011-01-16 00:04:22 UTC
rabbitmq-server-2.2.0-1.el5 has been submitted as an update for Fedora EPEL 5.
https://admin.fedoraproject.org/updates/rabbitmq-server-2.2.0-1.el5

Comment 8 Fedora Update System 2011-01-16 17:51:18 UTC
rabbitmq-server-2.2.0-1.el5 has been pushed to the Fedora EPEL 5 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update rabbitmq-server'.  You can provide feedback for this update here: https://admin.fedoraproject.org/updates/rabbitmq-server-2.2.0-1.el5

Comment 9 Fedora Update System 2011-02-07 17:53:46 UTC
rabbitmq-server-2.2.0-1.el5 has been pushed to the Fedora EPEL 5 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 10 Jose Pedro Oliveira 2011-04-14 01:13:02 UTC
Hubert,

Would it be possible to also build rabbitmq-server-2.2.0 for EPEL-6?
It appears to have been built only for F13, F14, F15, and EPEL-5.

tia,
jpo


Note You need to log in before you can comment on or make changes to this bug.