Description of problem: Currently, there is no way how to give test (through recipe) file. Having this file inside recipe gives way how to re-run test under same condition, currently, one can put URL into testparam to pass file, but this file may not be available in future. Having such options should give way how to e.g. temporarily update test (instead of pushing test to stable test repository, like updates.img in anaconda), attach data files to test (which will be available in future due to nature of recipe) and more. This is topic of my bachelor thesis, and I've already talked about this with Bill Peck.
Created attachment 834853 [details] beaker.patch This patch adds needed functionality to beaker to accept jobs XMLs with files as task "parameters", to store such files in database and to re-create job XML with those files. Relaxng scheme has documentation for all file parameters. There's currently missing documentation for "beaker.task.file.maxsize" config option which specifies size of stored file. Default value for this option is 0 effectively disabling files unless set otherwise. Only beaker-init should be needed for database modification.
Created attachment 834854 [details] simple job with file
Created attachment 834855 [details] job with base64 encoded file
Created attachment 834856 [details] job with large file for testing files excessing file limits
Created attachment 834857 [details] job using advanced features of file
Created attachment 834858 [details] beah.patch Patch for beah to parse file parameter from task and deploy such file on testing system. This implementation provides all file features mentioned in relaxng scheme for job and should be easy to extend. I've tested changes in beah on RHEL5.9 system which means python-2.4.
Job files in attachment should cover all basics around implementation of this feature, probably the most advanced feature is passing directories via tar.gz file encoded in base64 (in https://bugzilla.redhat.com/attachment.cgi?id=834857). I'm open to any discussion and concerns about proposed patches.
Thanks Pavel, this looks quite promising. The one point of concern we noted is that at the moment, the server components (in general) aren't very smart about the way they deal with large jobs. This can cause excessive memory usage for jobs that contain lots of recipe sets, recipes and tasks, and this feature is likely to make it even easier for users to get themselves into trouble. However, I don't consider that a *blocker* for incorporating this feature, just an existing problem that this feature will potentially make a bit worse. I've now added this issue to the backlog for 1.0 - we'll take a closer look at what will be involved in integrating it in the new year.
I see, I wasn't aware of this issue. Unfortunately this sounds to me like design issue, and nothing that I could do here to improve this, right?
Right, it's an existing design issue where there's a practical limit to how large a job definition can be before Beaker can't process it correctly. We won't let that stop us from incorporating this feature, though.
Hi Pavel, sorry for the extended period of silence on the patch. I've now been able to take a closer look, and while the general idea still seems sound, there are some key issues that would need to be addressed before it could be included in a release. - detailed review should take place on Gerrit rather than in Bugzilla. That means pushing the patch to gerrit.beaker-project.org, rather than just uploading the diff as an attachment (see http://beaker-project.org/dev/guide/writing-a-patch.html#submitting-your-patch for details) - the patch will need to be rebased against the current development branch. The largest issue with that is likely to be accounting for the fact bkr.server.model is now a package containing multiple submodules, rather than a single large Python file. - while the full provisioning cycle doesn't need to be tested (since it is up to the test or harness to actually read the files out of the XML), the patch should include tests for the XML processing code to ensure it accepts the new XML elements correctly, and that the new file elements are correctly duplicated when cloning a job.
Pavel, are you still interested in this feature? Would you be able to rebase the patch set and upload it to http://gerrit.beaker-project.org/ ? If you run into any issues you can ask here or ping us in #beaker.
Dear Pavel, another ping from me if you're still interested in this feature and push it up into gerrit?
Hello, I'll look if my patchset can be easily rebased this week. It will probably need some additional tuning (e.g. testsuite).
It's quite a long time since I worked on this and I didn't have a need for this feature it doesn't seem I'll have time for this in reasonable time frame. It doesn't seem I'll be rebasing it, so I'm closing this as WONTFIX. Feel free to reopen this bug or contact me if someone has a need for this feature.