Bug 1586187 - [RFE][S-3] Allow database dumps/backups to be uploaded to an FTP target
Summary: [RFE][S-3] Allow database dumps/backups to be uploaded to an FTP target
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Appliance
Version: 5.9.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: MVP
: 5.10.0
Assignee: Nick LaMuro
QA Contact: Jaroslav Henner
URL:
Whiteboard:
Depends On: 1586176
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-06-05 16:31 UTC by Nick LaMuro
Modified: 2019-02-07 23:03 UTC (History)
7 users (show)

Fixed In Version: 5.10.0.24
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-02-07 23:03:00 UTC
Category: ---
Cloudforms Team: CFME Core
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2019:0212 None None None 2019-02-07 23:03:08 UTC

Description Nick LaMuro 2018-06-05 16:31:48 UTC

Comment 2 Dave Johnson 2018-06-05 16:43:17 UTC
Please assess the impact of this issue and update the severity accordingly.  Please refer to https://bugzilla.redhat.com/page.cgi?id=fields.html#bug_severity for a reminder on each severity's definition.

If it's something like a tracker bug where it doesn't matter, please set the severity to Low.

Comment 5 Dave Johnson 2018-06-05 17:02:47 UTC
Please assess the impact of this issue and update the severity accordingly.  Please refer to https://bugzilla.redhat.com/page.cgi?id=fields.html#bug_severity for a reminder on each severity's definition.

If it's something like a tracker bug where it doesn't matter, please set the severity to Low.

Comment 7 Nick LaMuro 2018-09-19 16:52:25 UTC
Sorry for the delay in updates:

I have been actively working on this for a while, but it has been through a lot of re-writes based on other providers (s3 and swift) being added in parallel, which messed with my previous implementations.  Figured the thrashing of updates because of that wouldn't have been useful on this ticket.

* * *

Currently the following have been merged:

https://github.com/ManageIQ/manageiq-gems-pending/pull/360
https://github.com/ManageIQ/manageiq-gems-pending/pull/368

With the specifics of allowing FTP support to the rake tasks.  The only missing piece is the FTP options in the appliance console, which I will be adding shortly today (hopefully).  Will update when that patch is ready.


-Nick

Comment 8 Nick LaMuro 2018-09-19 19:45:30 UTC
Final PR for this feature can be found here (as promised in the above update):

https://github.com/ManageIQ/manageiq-appliance_console/pull/64


-Nick

Comment 9 Nick LaMuro 2018-09-20 18:04:19 UTC
Everything is now merged.

These features should be available in the next release of the `manageiq-appliance_console` gem, and I will update/finish this ticket when that will be available in the nightly builds.

Comment 11 CFME Bot 2018-11-06 15:47:32 UTC
New commit detected on ManageIQ/manageiq-appliance/master:

https://github.com/ManageIQ/manageiq-appliance/commit/9faaeedfd32b8456774519e6f09a156d852d8e6e
commit 9faaeedfd32b8456774519e6f09a156d852d8e6e
Author:     Nick Carboni <ncarboni@redhat.com>
AuthorDate: Mon Nov  5 14:14:02 2018 -0500
Commit:     Nick Carboni <ncarboni@redhat.com>
CommitDate: Mon Nov  5 14:14:02 2018 -0500

    Add console version 3.3

    Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1586176
    Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1586187
    Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1615488
    Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1632433

 manageiq-appliance-dependencies.rb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comment 12 CFME Bot 2018-11-08 17:22:08 UTC
New commit detected on ManageIQ/manageiq-appliance/hammer:

https://github.com/ManageIQ/manageiq-appliance/commit/b6b244d6b71cd26babca371c969e2447f1799ffa
commit b6b244d6b71cd26babca371c969e2447f1799ffa
Author:     Brandon Dunne <brandondunne@hotmail.com>
AuthorDate: Tue Nov  6 10:45:27 2018 -0500
Commit:     Brandon Dunne <brandondunne@hotmail.com>
CommitDate: Tue Nov  6 10:45:27 2018 -0500

    Merge pull request #215 from carbonin/use_version_3_3_of_the_console

    Add console version 3.3

    (cherry picked from commit ce975b028a06a43c4edc04fadf5a895015b4e874)

    Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1586176
    Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1586187
    Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1615488
    Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1632433

 manageiq-appliance-dependencies.rb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comment 13 Jaroslav Henner 2018-11-21 15:02:01 UTC
I can see this in when uploaded to ftp:

$ tree upload 
upload
└── db_dump
    └── tmp
        └── evm_db.dump

Should the "tmp" dir be there or should the evm_db.dump rather be in the db_dump dir directly?

I think the tmp directory is there as a part of the path on the appliance as I answered in the question:

Enter the location to save the dump file to: |/tmp/evm_db.dump|

Comment 14 Nick LaMuro 2018-11-26 18:52:17 UTC
This will require a bit of a convoluted explanation, but the short answer is "technically it is correct, but it is super confusing" (as I assume you are suggesting).

* * *

So for the long answer...

To start, originally the database restore action is what was original refactored out into the now "DatabaseAdmin" class, which was done here:

https://github.com/ManageIQ/manageiq-appliance_console/pull/42


Part the `/tmp/evm_db.backup` constant that is there as the default filename was originally in the bin file (still is):

https://github.com/ManageIQ/manageiq-appliance_console/blob/8959d6d6/bin/appliance_console#L54
https://github.com/ManageIQ/manageiq-appliance_console/blob/8959d6d6/lib/manageiq/appliance_console/database_admin.rb#L9-L10


Was ported over to the new DatabaseAdmin class, and was changed slightly when we were restoring a pg_dump versus a pg_basebackup.  The class then had functionality added to it to also handle the backup/dump functionality, since it effectively was doing the same thing for those actions as well.  But that said, the DatabaseAdmin class and menus are just a thin wrapper around the rake tasks in the main app:

https://github.com/ManageIQ/manageiq/blob/3ef8fd5a/lib/tasks/evm_dba.rake#L186-L192

which then call out the the `lib/evm_database_ops.rb` which have it's own file name defaulting:

https://github.com/ManageIQ/manageiq/blob/3ef8fd5a/lib/evm_database_ops.rb#L9-L10


along with code that adds the `db_dump` directory:

https://github.com/ManageIQ/manageiq/blob/3ef8fd5a/lib/evm_database_ops.rb#L125


That existed prior to me evening getting involved:

https://github.com/ManageIQ/manageiq/blob/705dae5f/lib/evm_database_ops.rb#L56



My approach with this all was to maintain as much as the previous functionality that was there previously to avoid breaking existing workflows, while still adding the new functionality and avoiding duplication along the way, emulating the existing features since there was nothing to go off of in the RFE.  Clearly, with this, there was some lapses in judgement from my end on what ends up being the best user experience, as well as a next to no direction on what the UI should look like as part of the request in this chain of RFEs.

Of note:  this behavior will happen with NFS/SMB/S3/Swift as well, so this really isn't related to this ticket (adding FTP support).


* * *

My suggestion, since this is just super confusing but not really a blocker, to open up a new ticket for the next release regarding this issue specifically.

We should have a much bigger discussion on how the defaulting should behave, and decide on if changing the defaults for the previously existing features is an acceptable breaking change to make this less confusing.  But, that discussion again should not happen here as it isn't directly related to the feature of FTP for backups/dumps/restores.

Comment 15 dmetzger 2018-11-26 20:31:15 UTC
Agree with comment #14 summary. The question raised is related to existing functionality and not something added / broken by this RFE, as such the feature is working as designed and an RFE BZ can be filed to enhance / simplify in the future.

Comment 16 Jaroslav Henner 2018-11-29 09:55:24 UTC
Thanks. It worked fine then.

Comment 17 errata-xmlrpc 2019-02-07 23:03:00 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:0212


Note You need to log in before you can comment on or make changes to this bug.