Description of problem: Uploading an image is complex process, requiring: - detecting the image type (qcow2, raw, iso) - creating a disk using the SDK, with the right format, sparse, initial_size and provional_size, which depend on the storage domain type - checking if the current host can be used for the transfer - starting a transfer on the current host if possible - if the transfer be started on the current host, the user have to choose the transfer_url or the proxy_url. - uploading the file using imageio REST API. - cancel transfer on failures We have example code and documentation for some of the steps, but building a correct upload application is too complicated. Even virt-v2v did not get it right yet, after few month of development, with help form the oVirt team. Uploading disks with several snapshots is even more complex. It does not make sense that all users will have to write this code and make the same errors. The upload and download examples in the SDK are usually wrong, outdated, and do not use the latest features offered by imageio. They do not have any automated tests so they are likely to break after every update. The upload and download examples also contain huge amount of duplicate code, and require too much time to maintain. Finally the ansible modules are using the outdated example code and need extra maintenance and testing. All these issues can be solved by providing a command line tool that does everything for you. Something like: ovirt-img upload --storage my-storage my-image.qcow2 https://example.com/ The command will: - read engine username and password from standard locations in a secure way - inspect the image properties and create disk property, matching the storage domain capabilities to the image type - inspect the current host and start the upload on it if posible - if the transfer cannot be started on current host, select the best transfer destination - upload the data efficiently, supporting sparse images - handle failures properly, avoiding leaving paused transfers - support resuming if the upload was aborted - show proper progress during upload The data path part is already implemented in https://gerrit.ovirt.org/c/94216/
This bug has not been marked as blocker for oVirt 4.3.0. Since we are releasing it tomorrow, January 29th, this bug has been re-targeted to 4.3.1.
(In reply to Nir Soffer from comment #0) > Description of problem: > > Uploading an image is complex process, requiring: > - detecting the image type (qcow2, raw, iso) > - creating a disk using the SDK, with the right format, sparse, initial_size > and provional_size, which depend on the storage domain type > - checking if the current host can be used for the transfer > - starting a transfer on the current host if possible > - if the transfer be started on the current host, the user have to choose the > transfer_url or the proxy_url. > - uploading the file using imageio REST API. > - cancel transfer on failures > > We have example code and documentation for some of the steps, but building > a correct upload application is too complicated. Even virt-v2v did not get > it > right yet, after few month of development, with help form the oVirt team. > > Uploading disks with several snapshots is even more complex. > > It does not make sense that all users will have to write this code and make > the > same errors. > > The upload and download examples in the SDK are usually wrong, outdated, and > do > not use the latest features offered by imageio. They do not have any > automated > tests so they are likely to break after every update. > > The upload and download examples also contain huge amount of duplicate code, > and > require too much time to maintain. > > Finally the ansible modules are using the outdated example code and need > extra > maintenance and testing. All oVirt teams have responsibility to make oVirt Ansible modules up-to-date regarding to oVirt features. The command line tool is really required, but I think we should really provide the same solution even for oVirt Ansible modules, so it would really great it you would add it to this RFE (either as an update to ovirt_disk module or newly created ovirt_image module)
(In reply to Martin Perina from comment #2) > All oVirt teams have responsibility to make oVirt Ansible modules up-to-date > regarding to oVirt features. The command line tool is really required, but I > think we should really provide the same solution even for oVirt Ansible > modules, so it would really great it you would add it to this RFE (either as > an update to ovirt_disk module or newly created ovirt_image module) With the new command line tool, the ansible module will be much simpler, just call the command with the right arguments. No need to maintain duplicate code in different languages.
Since this was opened, we now have new backup functionality in: https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/backup_vm.py I think backup and incremental backup are clear candidates for this tool, since restore is mainly upload operation. We also have a new package - ovirt-imageio-client; the command line tool seems to fit in this package. Having a command line tool, we can enhance it later to upload or download ova files in a smart way, for example, extract images from ova files and upload them transparently, or download images and vm information and pack them in ova file. Another possible extension, upload and download images from Glance. Currently implemented in vdsm but can be much better as command line tool that vdsm can run if needed, or user can run on any host.
Proposing for 4.4.2 since this is important features for users and backup vendors.
Give testing_ack for this RFE, but there isn't a "devel_ack" here.
Shir, no acks needed in ovirt bugs.
due to constant arguments about sdk examples this still makes a lot of sense to create. even if still unsupported (e.g in vdsm's contrib/ or somewhere)
We are past 4.5.0 feature freeze, please re-target.
Moved to GitHub: https://github.com/oVirt/ovirt-imageio/issues/72