Description of problem: When cloning a job which was provisioned with the new <distro /> tag from 911515 and downgrading/upgrading the Beaker database I'll get a traceback: https://bugzilla.redhat.com/show_bug.cgi?id=911515#c44 Version-Release number of selected component (if applicable): develop HEAD How reproducible: 100% Steps to Reproduce: 1. Install latest nightly build 2. In the WebUI scheduled a job using the <distro /> tag 3. Immediately abort the job (since running it is not important for this bug) 4. Stop all beaker services 5. Downgrade the database: beaker-init --debug --downgrade=24.5 6. Upgrade the database: beaker-init --debug 7. Start all services 8. In the web UI clone the previously created job from 2) Actual results: Traceback Expected results: No traceback Additional info:
This may be difficult/impossible to fix. I am guessing that, right now, on downgrade the relevant new columns are being dropped (thus their data is lost) and on upgrade the columns are re-created and left with NULLs. This leaves the job in a kind of in-between state, where the code thinks it looks like a job using <distro/> but some of the data has since been lost. You have some choices: * Don't drop the columns on downgrade, leave them in place, so that the data is not lost. The upgrade then has to deal with the possibility that the columns may or may not exist and do the correct thing in both cases. (In theory we should write every single migration like this, but in practice we don't because of the difficulty.) * Fix the downgrade process to *fully* "destroy" the data on the new job. That is, make the job using <distro/> look like a totally normal old-fashioned job which used <distroRequires/>. Then after upgrade, the code should treat it as a job which used <distroRequires/>. The current migration actually attempts this (by assigning distro_tree_id = 1) but I am guessing it is not completely filling in the necessary columns to convert the job into an old one. In particular I bet you that _distro_requires is not filled in with some sensible XML. * Call it WONTFIX and declare that, if you start using the new <distro/> functionality, then you cannot downgrade your Beaker instance anymore.
(In reply to Dan Callaghan from comment #2) > This may be difficult/impossible to fix. I am guessing that, right now, on > downgrade the relevant new columns are being dropped (thus their data is > lost) and on upgrade the columns are re-created and left with NULLs. This > leaves the job in a kind of in-between state, where the code thinks it looks > like a job using <distro/> but some of the data has since been lost. That is indeed what happens https://gerrit.beaker-project.org/#/c/5921/25/Server/bkr/server/alembic/versions/3ba776df4c76_add_distro_metadata_columns_to_installation.py
We discussed this as a team. Let's see how point a) would work out and how complex this would be. We can always go for the WONTFIX solution.
Tested this as follows: Pre-patch: Created a job with <distro> tag. Shut down sever. Downgraded to 24.5 using beaker-init Verified that the database columns in the installation table were dropped. Upgraded to latest using beaker-init. Restarted server Got stack trace when cloning a job. This patch: Created a new job with <distro> tag. Shut down server Downgraded to 24.5 using beaker-init Verified that the newly created database columns were NOT dropped. Upgraded to latest version using beaker-init. Restarted server. Cloned job successfully. Marking as Verified.
Beaker 25.0 has been released. Release notes are available upstream: https://beaker-project.org/docs/whats-new/release-25.html