Bug 1605016

Summary: vagrant up crashes
Product: [Fedora] Fedora Reporter: Anatoli Babenia <anatoli>
Component: vagrantAssignee: Pavel Valena <pvalena>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 28CC: egegunes, lmohanty, madam, pvalena, strzibny, thrcka, vondruch
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: vagrant-2.0.2-2.fc28 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-07-31 18:02:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Fix applied - log from testing none

Description Anatoli Babenia 2018-07-20 01:40:53 UTC
Description of problem:
Unable to run dev env for https://github.com/fedora-infra/fedora-packages

Version-Release number of selected component (if applicable): vagrant-2.0.2-1.fc28

How reproducible: 100%

Steps to Reproduce: 

1. dnf install vagrant
2. vagrant up

Actual results:

 ✗ vagrant up      
#<Thread:0x000056470dda7af0@/usr/share/vagrant/gems/gems/vagrant-2.0.2/lib/vagrant/util/checkpoint_client.rb:76 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
	2541: from /usr/share/vagrant/gems/gems/vagrant-2.0.2/lib/vagrant/util/checkpoint_client.rb:78:in `block in check'
	2540: from /usr/share/gems/gems/hashicorp-checkpoint-0.1.5/lib/checkpoint.rb:91:in `check'
	2539: from /usr/share/ruby/net/http.rb:1213:in `get'
	2538: from /usr/share/ruby/net/http.rb:1455:in `request'
	2537: from /usr/share/ruby/net/http.rb:909:in `start'
	2536: from /usr/share/ruby/net/http.rb:920:in `do_start'
	2535: from /usr/share/ruby/net/http.rb:935:in `connect'
	2534: from /usr/share/ruby/timeout.rb:103:in `timeout'
	 ... 2529 levels...
	   4: from /usr/share/ruby/resolv.rb:769:in `sender'
	   3: from /usr/share/ruby/resolv.rb:629:in `allocate_request_id'
	   2: from /usr/share/ruby/resolv.rb:629:in `synchronize'
	   1: from /usr/share/ruby/resolv.rb:630:in `block in allocate_request_id'
/usr/share/ruby/resolv.rb:630:in `hash': stack level too deep (SystemStackError)
Traceback (most recent call last):
	2541: from /usr/share/vagrant/gems/gems/vagrant-2.0.2/lib/vagrant/util/checkpoint_client.rb:78:in `block in check'
	2540: from /usr/share/gems/gems/hashicorp-checkpoint-0.1.5/lib/checkpoint.rb:91:in `check'
	2539: from /usr/share/ruby/net/http.rb:1213:in `get'
	2538: from /usr/share/ruby/net/http.rb:1455:in `request'
	2537: from /usr/share/ruby/net/http.rb:909:in `start'
	2536: from /usr/share/ruby/net/http.rb:920:in `do_start'
	2535: from /usr/share/ruby/net/http.rb:935:in `connect'
	2534: from /usr/share/ruby/timeout.rb:103:in `timeout'
	 ... 2529 levels...
	   4: from /usr/share/ruby/resolv.rb:769:in `sender'
	   3: from /usr/share/ruby/resolv.rb:629:in `allocate_request_id'
	   2: from /usr/share/ruby/resolv.rb:629:in `synchronize'
	   1: from /usr/share/ruby/resolv.rb:630:in `block in allocate_request_id'
/usr/share/ruby/resolv.rb:630:in `hash': stack level too deep (SystemStackError)

Comment 1 Anatoli Babenia 2018-07-20 02:00:46 UTC
Upstream bug https://github.com/hashicorp/vagrant/issues/9595 supposedly fixed in 2.0.4 in https://github.com/hashicorp/vagrant/pull/9644

Comment 2 Pavel Valena 2018-07-20 13:33:27 UTC
Hello Anatoli,

thank you for your report. Could you paste your Vagrantfile and `/etc/resolv.conf`?

Also, you could try running:
```
export VAGRANT_DISABLE_RESOLV_REPLACE=1

```

as mentioned in the upstream issue #9595.

Thanks!

Comment 3 Anatoli Babenia 2018-07-20 14:35:28 UTC
✗ cat /etc/resolv.conf 
# Generated by NetworkManager
nameserver fe80::1%wlp3s0
nameserver 192.168.0.1
nameserver 8.8.8.8



✗ cat Vagrantfile 
# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure(2) do |config|
  config.vm.box_url = "https://download.fedoraproject.org/pub/fedora/linux/releases/25/CloudImages/x86_64/images/Fedora-Cloud-Base-Vagrant-25-1.3.x86_64.vagrant-libvirt.box"
  config.vm.box = "f25-cloud-libvirt"
  config.vm.network "forwarded_port", guest: 8080, host: 8080
  config.vm.synced_folder ".", "/vagrant", type: "sshfs"
  # Ansible needs the guest to have these
  config.vm.provision "shell", inline: "sudo dnf install -y libselinux-python python2-dnf"

  config.vm.provision "ansible" do |ansible|
      ansible.playbook = "devel/ansible/playbook.yml"
  end
end

Comment 4 Anatoli Babenia 2018-07-20 14:52:03 UTC
After export VAGRANT_DISABLE_RESOLV_REPLACE=1 there is no crash anymore, but another issue with downloaded box - https://pastebin.mozilla.org/9090160

Comment 5 Pavel Valena 2018-07-20 15:38:10 UTC
Thanks. I'll make a reproducer and push update to F28.

The latter one is probably an unrelated one. Let me check later on.

Comment 6 Anatoli Babenia 2018-07-20 16:07:50 UTC
For the latter bug.

    git clone https://github.com/fedora-infra/fedora-packages
    cd fedora-packages
    vagrant up

https://github.com/fedora-infra/fedora-packages/issues/394

Comment 7 Pavel Valena 2018-07-20 16:46:57 UTC
I've failed to reproduce the issue insofar.

```
$ head -1 /etc/resolv.conf        # this is probably triggring the issue
nameserver fe80::1%enp0s25
$ vagrant init -fm fedora/28-cloud-base
$ vagrant up
 [ . . . ]
==> default: Successfully added box 'fedora/28-cloud-base' (v20180425) for 'libvirt'!
 [ . . . ]
==> default: Configuring and enabling network interfaces...
$ vagrant ssh -c 'echo TEST'
TEST
Connection to 192.168.121.239 closed.
```

Could you please try this build?
  https://koji.fedoraproject.org/koji/taskinfo?taskID=28472515


Regarding the second issue, you can work around that by adding the box manually, f.e.:
  $ vagrant box add --name f28-cloud-libvirt --location-trusted https://download.fedoraproject.org/pub/fedora/linux/releases/28/Cloud/x86_64/images/Fedora-Cloud-Base-Vagrant-28-1.1.x86_64.vagrant-libvirt.box

Comment 8 Pavel Valena 2018-07-20 16:51:23 UTC
OK, it got triggered right after submitting my previous comment.

I can confirm it fixes the issue. I'll create the update then.

Comment 9 Pavel Valena 2018-07-20 17:40:25 UTC
Created attachment 1466887 [details]
Fix applied - log from testing

The first issue does seem to be 100% reproducible.

Also, note that I did not trigger the second issue(log attached). It could have been triggered by some other circumstances(`==> default: Box download is resuming from prior download progress`).
Please create an another bug for that if that other issue persists.

Comment 10 Fedora Update System 2018-07-21 00:48:44 UTC
vagrant-2.0.2-2.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-7a88c08af8

Comment 11 Anatoli Babenia 2018-07-21 01:26:15 UTC
Why not 2.0.4? https://github.com/hashicorp/vagrant/blob/v2.1.2/CHANGELOG.md#204-april-20-2018 If the package is not usable in its current form, and is a minor bug fix version, why not to pull all other stable fixes?

Comment 12 Pavel Valena 2018-07-21 03:20:43 UTC
(In reply to Anatoli Babenia from comment #11)
> Why not 2.0.4?
> https://github.com/hashicorp/vagrant/blob/v2.1.2/CHANGELOG.md#204-april-20-
> 2018 If the package is not usable in its current form, and is a minor bug
> fix version, why not to pull all other stable fixes?

Good point.

The reason is that this would currently require more extensive checks/testing than we have capacity for. You can however do the checks / review:
  https://src.fedoraproject.org/rpms/vagrant/pull-request/10

Comment 13 Fedora Update System 2018-07-22 03:39:01 UTC
vagrant-2.0.2-2.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-7a88c08af8

Comment 14 Anatoli Babenia 2018-07-22 03:59:21 UTC
The problem with downloaded images is caused by my `bsdtar` being 0 length - https://ask.fedoraproject.org/en/question/124355/using-dnf-to-compare-filesystem-contents-with-repositories/

    ✗ vagrant box add Fedora-Cloud-Base-Vagrant-25-1.3.x86_64.vagrant-libvirt.box --name f25-cloud-libvirt --debug

I do not run vagrant with sudo, so it could not overwrite this. But there is ongoing kernel bug https://bugzilla.redhat.com/show_bug.cgi?id=1598462 that could result in this filesystem corruption.

Once I check my filesystem, I will be able to get back to testing 2.0.4 build.

Comment 15 Fedora Update System 2018-07-31 18:02:39 UTC
vagrant-2.0.2-2.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.