Description of problem: Provide admin the ability to set an S3/Swift endpoint as a valid backup destination for the CFME database. Version-Release number of selected component (if applicable): Running on CloudForms 4.5 at this time. CFME Version: 5.8.2.3.20171016155816_aaec796 How reproducible: Currently only CIFS/NFS are supported as backup destinations. Steps to Reproduce: 1. Setup CFME Backup to NFS Storage Actual results: No S3/Swift endpoint feasible. Expected results: Ideally add in the ability to set a URI, user, and key to push backups directly into a bucket. Additional info: RHN Case: 01974285
Please assess the impact of this issue and update the severity accordingly. Please refer to https://bugzilla.redhat.com/page.cgi?id=fields.html#bug_severity for a reminder on each severity's definition. If it's something like a tracker bug where it doesn't matter, please set the severity to Low.
This is a feature request that was opened by a customer, who has also opened a Support case (linked above) for this RFE as well. I have linked the RHN Case to this BZ. I have set the priority and severity to mimic the priority as specified in the support case.
https://github.com/ManageIQ/manageiq-gems-pending/pull/355
https://github.com/ManageIQ/manageiq-gems-pending/pull/357
PRs being worked for this include https://github.com/ManageIQ/manageiq-gems-pending/pull/357 (S3 Backup) https://github.com/ManageIQ/manageiq-ui-classic/pull/4300 (S3 UI Changes) https://github.com/ManageIQ/manageiq/pull/17689 (WIP but changing that today) https://github.com/ManageIQ/manageiq-schema/pull/227 (added region to file_depot) https://github.com/ManageIQ/manageiq-gems-pending/pull/355 (S3 Restore) Still working on / testing manageiq-appliance_console changes for a separate PR, and will need to do Swift changes as well still although those should be *incredibly* similar to work done already and should be very fast.
New commit detected on ManageIQ/manageiq-gems-pending/master: https://github.com/ManageIQ/manageiq-gems-pending/commit/65ee18925b687af0289f9d3f588b4b05184577ce commit 65ee18925b687af0289f9d3f588b4b05184577ce Author: Jerry Keselman <jkeselma> AuthorDate: Tue Jun 26 10:18:02 2018 -0400 Commit: Jerry Keselman <jkeselma> CommitDate: Tue Jun 26 10:18:02 2018 -0400 Support S3 for DB Backups Initial commit to support DB backups to S3. Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1513520 lib/gems/pending/util/mount/miq_generic_mount_session.rb | 9 +- lib/gems/pending/util/mount/miq_s3_session.rb | 66 + 2 files changed, 74 insertions(+), 1 deletion(-)
This should not be on POST yet. Still more PRs upcoming. https://github.com/ManageIQ/manageiq-gems-pending/pull/357 https://github.com/ManageIQ/manageiq-gems-pending/pull/355 https://github.com/ManageIQ/manageiq-schema/pull/227 and https://github.com/ManageIQ/manageiq/pull/17689 have been merged. https://github.com/ManageIQ/manageiq-ui-classic/pull/4300 is being reviewed.
https://github.com/ManageIQ/manageiq/pull/17791 has been merged as well. https://github.com/ManageIQ/manageiq-ui-classic/pull/4300 is still being reviewed. Another PR against manageiq-appliance_console still to be added.
An addition PR - https://github.com/ManageIQ/manageiq/pull/17803 is required to save the AWS region in the DB.
https://github.com/ManageIQ/manageiq-appliance_console/pull/54 and https://github.com/ManageIQ/manageiq/pull/17827 Are the last two PRs to be added for the S3 portion of this RFE. https://github.com/ManageIQ/manageiq/pull/17803 has been merged. https://github.com/ManageIQ/manageiq-ui-classic/pull/4300 has been reviewed and we are waiting for signoff from second UI developer for merge approval. Other than any changes to the above PRs pending review development is complete for the S3 portion of this RFE and the Openstack Swift portion can now commence.
All PRs above required for DB Backups and Restores via AWS S3 are merged.
I am going to split this RFE in two - this BZ will track the AWS S3 changes already completed, and the new one will track the Openstack Swift changes. The new RFE BZ for Swift is https://bugzilla.redhat.com/show_bug.cgi?id=1615488. Moving this current BZ to POST.
Moving back to POST as appliance_console changes are released.
Nick is currently working to fix the restore issue across all filesystem / object store types. Reassigning this to him until it is fixed.
partial fix: https://github.com/ManageIQ/manageiq/pull/18144
New commit detected on ManageIQ/manageiq/hammer: https://github.com/ManageIQ/manageiq/commit/92dd1890250e22e705503601ae2f5397faf3b842 commit 92dd1890250e22e705503601ae2f5397faf3b842 Author: Nick Carboni <ncarboni> AuthorDate: Mon Nov 5 17:25:17 2018 -0500 Commit: Nick Carboni <ncarboni> CommitDate: Mon Nov 5 17:25:17 2018 -0500 Merge pull request #18144 from NickLaMuro/fix_miq_file_storage_send_args_for_download Fix send_args for EvmDatabaseOps.restore (cherry picked from commit 1775555cfe823ffd5c3c3c50137761c43cf49827) https://bugzilla.redhat.com/show_bug.cgi?id=1633573 https://bugzilla.redhat.com/show_bug.cgi?id=1513520 lib/evm_database_ops.rb | 7 + spec/lib/evm_database_ops_spec.rb | 2 +- 2 files changed, 8 insertions(+), 1 deletion(-)
The following have been merged: https://github.com/ManageIQ/manageiq-gems-pending/pull/385 https://github.com/ManageIQ/manageiq-gems-pending/pull/389 https://github.com/ManageIQ/manageiq-gems-pending/pull/390 https://github.com/ManageIQ/manageiq-gems-pending/pull/395 https://github.com/ManageIQ/manageiq-gems-pending/pull/396 https://github.com/ManageIQ/manageiq-gems-pending/pull/397 https://github.com/ManageIQ/manageiq-gems-pending/pull/398 https://github.com/ManageIQ/manageiq-gems-pending/pull/399 https://github.com/ManageIQ/manageiq-gems-pending/pull/400 https://github.com/ManageIQ/manageiq-gems-pending/pull/401 https://github.com/ManageIQ/manageiq-gems-pending/pull/402 https://github.com/ManageIQ/manageiq-gems-pending/pull/405 https://github.com/ManageIQ/manageiq-gems-pending/pull/406 https://github.com/ManageIQ/manageiq-gems-pending/pull/410 https://github.com/ManageIQ/manageiq/pull/18144 https://github.com/ManageIQ/manageiq/pull/18166 https://github.com/ManageIQ/manageiq/pull/18167 The PRs specifically related to s3 from above are: https://github.com/ManageIQ/manageiq-gems-pending/pull/397 https://github.com/ManageIQ/manageiq-gems-pending/pull/399 Which should now fix the issue reported (and further ones not reported as well) regarding database restores.
Nick, I tested it and when evmserverd is stopped it works properly, but it doesn't work when it's running. Should it be able to restore database while evmserverd is running? Thanks
Matous, I don't believe we have a document specifically for DB restore procedure but it is expected for the evmeserverd service to be stopped. We don't support DB restore while the service is running. (removing needinfo)
Jan, so there should be text indicating that evmserverd must be turned off before applying restore. Or it should check whether evmserverd is running? I know this is minor issue so I can create another RFE BZ for that.
Yes, an RFE for a message makes sense here, Matous. Definitely open it and let's see; worst-case scenario - it gets closed during triage.
Verified in 5.10.0.28. Both backup and restore are working and also created separate BZ restoring backup while evmserverd is running: https://bugzilla.redhat.com/show_bug.cgi?id=1662991
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2019:0212