Bug 1474353

Summary: Vagrant fails to "up" machine after its storage image manually removed.
Product: [Fedora] Fedora Reporter: Julius Milan <jmilan>
Component: libvirtAssignee: Libvirt Maintainers <libvirt-maint>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 25CC: agedosier, berrange, clalancette, crobinso, dima85, itamar, laine, libvirt-maint, lmohanty, madam, phrdina, pvalena, strzibny, veillard, virt-maint, vondruch
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-09-14 20:27:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
used Vagrantfile none

Description Julius Milan 2017-07-24 12:45:01 UTC
Created attachment 1303627 [details]
used Vagrantfile

Description of problem:

Vagrant fails to "up" the machine after it is destroyed and its storage image manually removed.
Using the attached Vagrantfile error occurred after series of steps described in "How reproducible" part.


Version-Release number of selected component (if applicable):

vagrant-libvirt-0.0.35-4.fc25.noarch
vagrant-1.8.5-3.fc25.noarch
(Not tested on F26)


How reproducible:
always


Steps to Reproduce:

1. vagrant up                                                        # (Vagrantfile attached)
2. vagrant destroy
3. rm ~/vm_rhel_storage/f25-cloud-libvirt_vagrant_box_image_0.img    # remove img created by first "vagrant up"
4. vagrant up


Actual results:

$ sudo vagrant up
Bringing machine 'nuancier' up with 'libvirt' provider...
==> nuancier: Uploading base box image as volume into libvirt storage...
/usr/share/gems/gems/fog-libvirt-0.3.0/lib/fog/libvirt/requests/compute/create_volume.rb:6:in `create_volume_xml': Call to virStorageVolCreateXML failed: storage volume 'f25-cloud-libvirt_vagrant_box_image_0.img' exists already (Libvirt::Error)
	from /usr/share/gems/gems/fog-libvirt-0.3.0/lib/fog/libvirt/requests/compute/create_volume.rb:6:in `create_volume'
	from /usr/share/gems/gems/fog-libvirt-0.3.0/lib/fog/libvirt/models/compute/volume.rb:40:in `save'
	from /usr/share/gems/gems/fog-core-1.34.0/lib/fog/core/collection.rb:51:in `create'
	from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.0.35/lib/vagrant-libvirt/action/handle_box_image.rb:78:in `block in call'
	from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.0.35/lib/vagrant-libvirt/action/handle_box_image.rb:60:in `synchronize'
	from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.0.35/lib/vagrant-libvirt/action/handle_box_image.rb:60:in `call'
	from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
	from /usr/share/vagrant/lib/vagrant/action/builtin/handle_box.rb:56:in `call'
	from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
	from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.0.35/lib/vagrant-libvirt/action/handle_storage_pool.rb:50:in `call'
	from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
	from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.0.35/lib/vagrant-libvirt/action/set_name_of_domain.rb:35:in `call'
	from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
	from /usr/share/vagrant/lib/vagrant/action/warden.rb:95:in `block in finalize_action'
	from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
	from /usr/share/vagrant/lib/vagrant/action/builder.rb:116:in `call'
	from /usr/share/vagrant/lib/vagrant/action/runner.rb:66:in `block in run'
	from /usr/share/vagrant/lib/vagrant/util/busy.rb:19:in `busy'
	from /usr/share/vagrant/lib/vagrant/action/runner.rb:66:in `run'
	from /usr/share/vagrant/lib/vagrant/action/builtin/call.rb:53:in `call'
	from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
	from /usr/share/vagrant/lib/vagrant/action/builtin/config_validate.rb:25:in `call'
	from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
	from /usr/share/vagrant/lib/vagrant/action/builder.rb:116:in `call'
	from /usr/share/vagrant/lib/vagrant/action/runner.rb:66:in `block in run'
	from /usr/share/vagrant/lib/vagrant/util/busy.rb:19:in `busy'
	from /usr/share/vagrant/lib/vagrant/action/runner.rb:66:in `run'
	from /usr/share/vagrant/lib/vagrant/machine.rb:225:in `action_raw'
	from /usr/share/vagrant/lib/vagrant/machine.rb:200:in `block in action'
	from /usr/share/vagrant/lib/vagrant/environment.rb:561:in `lock'
	from /usr/share/vagrant/lib/vagrant/machine.rb:186:in `call'
	from /usr/share/vagrant/lib/vagrant/machine.rb:186:in `action'
	from /usr/share/vagrant/lib/vagrant/batch_action.rb:82:in `block (2 levels) in run'

But works nicely after reboot.


Expected results:
To "up" the machine anyway.


Additional info:

Comment 1 Pavel Valena 2017-07-24 13:29:36 UTC
Hello Julius,

I can confirm this occurs with any Vagrantfile I've tried and also on F26.

This can be worked around by restarting libvirt daemon.

```
systemctl restart libvirtd

```

Also when I remove the box together with the imported libvirt image the issue persists, so it is cached by libvirt daemon. Thus this is probably a libvirt issue.

Comment 2 Pavel Hrdina 2017-07-24 14:11:21 UTC
I don't think that this is an issue in libvirt.  It's wrong usage.

Vagrant creates that image in libvirt's some storage pool.  Since libvirt
doesn't use inotify, and we are not planning it using, libvirt cannot now that
you've removed the file itself and it still has a volume entry for that
storage pool.  You can check it by running

  "virsh vol-list $pool-name"

after you remove the file.

Instead of executing 

  "rm ~/vm_rhel_storage/f25-cloud-libvirt_vagrant_box_image_0.img"

you should run

  "virsh vol-delete f25-cloud-libvirt_vagrant_box_image_0.img $pool-name"

If you want to still use the "rm" command, you can run

  "virsh pool-refresh $pool-name"

to detect that the image was removed.

Generally you shouldn't modify files that are handled by libvirt behind
libvirt's back.  Sometimes it's possible, like with the "virsh pool-refresh"
which can re-detect the changes, but for other cases there is no recovery from
that.

Comment 3 Vít Ondruch 2017-07-24 14:33:53 UTC
I agree with Pavel's observations and I tend to reject this as a NOTABUG.

However, to improve the user experience (since the libvirt volumes are not the easiest thing to grasp for newcomers and they tend to consume significant disk space), @Pavel do you think it would be reasonable for vagrant-libvirt to call something like "virsh pool-refresh $pool-name" for example every time "vagrant up" is called, to ensure that libvirt storage is consistent? Because if the volume was removed using "virsh vol-delete", vagrant-libvirt would handle such situation just fine. But this would be upstream issue anyway ...

Comment 4 Pavel Valena 2017-07-24 15:15:39 UTC
Note that vagrant-libvirt is checking for the image in volumes list like so[1] in several places, but does not introduce any logics in doing so, but merely delegating all operations to Fog[2].
Therefore, from my POV, the logic does not fit in vagrant-libvirt scope.

[1] https://github.com/vagrant-libvirt/vagrant-libvirt/blob/c1898be3d6383a88b9a2408e81a8947e615edf5a/lib/vagrant-libvirt/action/handle_box_image.rb#L62
[2] https://github.com/vagrant-libvirt/vagrant-libvirt/blob/c1898be3d6383a88b9a2408e81a8947e615edf5a/lib/vagrant-libvirt/driver.rb#L41

Comment 5 Cole Robinson 2017-09-14 20:27:24 UTC
Doesn't sound like there's a libvirt bug here. Note there's been an RFE for a while for libvirt to basically detect pool=dir file changes automatically, if that's every deemed feasible and implemented it would make this case 'just work':

https://bugzilla.redhat.com/show_bug.cgi?id=821508