Description of problem: You can't 'docker pull' in a virt-builder environment. I suspect this is because something needed by the docker daemon is missing in the virt-builder environment. As an aside, I think it's ridiculous that you need a daemon running to do a docker pull, but fixing that is out of scope for this bug report :( Version-Release number of selected component (if applicable): $ virt-builder --version virt-builder 1.28.6 (on Fedora 21) How reproducible: 100% Steps to Reproduce: $ virt-builder fedora-21 --install docker --run-command 'docker pull fedora:21' [ 8.0] Downloading: http://libguestfs.org/download/builder/fedora-21.xz [ 8.0] Planning how to build this image [ 8.0] Uncompressing [ 25.0] Opening the new disk [ 60.0] Setting a random seed [ 60.0] Installing packages: docker [ 154.0] Running: docker pull fedora:21 ftp://ftp.cogeco.net/fedora/linux/releases/21/Everything/x86_64/os/repodata/4c0ea0d0ca8fd81fd3a96cacabfbcf9e02c33125670505fcf20aacefab48df02-primary.sqlite.xz: [Errno 12] Timeout on ftp://ftp.cogeco.net/fedora/linux/releases/21/Everything/x86_64/os/repodata/4c0ea0d0ca8fd81fd3a96cacabfbcf9e02c33125670505fcf20aacefab48df02-primary.sqlite.xz: (28, '') Trying other mirror. Resolving Dependencies --> Running transaction check ---> Package docker-io.x86_64 0:1.5.0-1.fc21 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: docker-io x86_64 1.5.0-1.fc21 updates 5.8 M Transaction Summary ================================================================================ Install 1 Package Total download size: 5.8 M Installed size: 26 M Downloading packages: warning: /var/cache/yum/x86_64/21/updates/packages/docker-io-1.5.0-1.fc21.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 95a43f54: NOKEY Public key for docker-io-1.5.0-1.fc21.x86_64.rpm is not installed Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-21-x86_64 Importing GPG key 0x95A43F54: Userid : "Fedora (21) <fedora>" Fingerprint: 6596 b8fb abda 5227 a9c5 b59e 89ad 4e87 95a4 3f54 Package : fedora-repos-21-2.noarch (@anaconda) From : /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-21-x86_64 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : docker-io-1.5.0-1.fc21.x86_64 1/1 Verifying : docker-io-1.5.0-1.fc21.x86_64 1/1 Installed: docker-io.x86_64 0:1.5.0-1.fc21 Complete! time="2015-02-24T13:49:03-05:00" level="fatal" msg="Post http:///var/run/docker.sock/v1.17/images/create?fromImage=fedora%3A21: dial unix /var/run/docker.sock: no such file or directory" virt-builder: error: docker pull fedora:21: command exited with an error If reporting bugs, run virt-builder with debugging enabled and include the complete output: virt-builder -v -x [...] Actual results: fail. Expected results: win. Additional info: I didn't attach the long debug output because this is an easy reproducer. If you can't reproduce, let me know, and I'll attach. Cheers!
The following *nearly* worked for me (and yes, the 'sleep' command *is* necessary, grumble grumble): virt-builder fedora-21 \ --install docker \ --run-command ' docker -H unix:///var/run/docker.sock -d & sleep 5 docker pull fedora:21 ' It eventually prints: [...] 511136ea3c5a: Pulling fs layer 00a0c78eeb6d: Pulling fs layer 834629358fe2: Pulling fs layer 511136ea3c5a: Verifying Checksum 511136ea3c5a: Download complete 00a0c78eeb6d: Verifying Checksum 00a0c78eeb6d: Download complete 511136ea3c5a: Pull complete 00a0c78eeb6d: Pull complete followed by an error, presumably because 'docker pull' exits with the wrong error code, or doesn't print an error message. You could add ||: after the docker pull command to ignore that. Proxy settings from outside virt-builder are, however, not passed through to docker, although as far as I can see that is a docker problem, not a virt-builder problem. Similar to: https://github.com/tmatilai/vagrant-proxyconf/issues/97
Hrm. I ran that and here is what I got: $ virt-builder fedora-21 \ > --install docker \ > --run-command ' > docker -H unix:///var/run/docker.sock -d & > sleep 5 > docker pull fedora:21 > ' [ 4.0] Downloading: http://libguestfs.org/download/builder/fedora-21.xz [ 5.0] Planning how to build this image [ 5.0] Uncompressing [ 17.0] Opening the new disk [ 40.0] Setting a random seed [ 40.0] Installing packages: docker [ 102.0] Running: docker -H unix:///var/run/docker.sock -d & sleep 5 docker pull fedora:21 [ 621.0] Setting passwords virt-builder: Setting random password of root to 7II5KPoA7uHByrt1 [ 622.0] Finishing off virt-builder: error: libguestfs error: umount_all: umount: /sysroot: umount: /sysroot: target is busy (In some cases useful info about processes that use the device is found by lsof(8) or fuser(1).) If reporting bugs, run virt-builder with debugging enabled and include the complete output: virt-builder -v -x [...] I didn't dig too deeply, but is this sysroot error caused by something that the daemon is still holding on to? What about the sleep 5, why is this necessary? Is there a clean solution, or a bug I can file with the docker people? Cheers
Yup I suspect the docker daemon is holding /sysroot open. Probably adding `killall docker' as the last command could fix that. Or sending a "terminate the daemon" command if there is one. The sleep 5 was necessary for me - it could be that docker doesn't create the socket before forking into the background (which would be a bug in docker if true).
I've created a docker issue to see if those folks can help suggest a solution or a fix, so that this works together more elegantly. https://github.com/docker/docker/issues/11241
$ cat vb.sh #!/bin/bash virt-builder fedora-21 \ --install docker \ --run-command ' docker -H unix:///var/run/docker.sock -d & sleep 5 docker pull fedora:21 killall docker ' $ ./vb.sh [ 2.0] Downloading: http://libguestfs.org/download/builder/fedora-21.xz [ 2.0] Planning how to build this image [ 2.0] Uncompressing [ 14.0] Opening the new disk [ 29.0] Setting a random seed [ 29.0] Installing packages: docker [ 62.0] Running: docker -H unix:///var/run/docker.sock -d & sleep 5 docker pull fedora:21 killall docker ftp://ftp.cogeco.net/fedora/linux/releases/21/Everything/x86_64/os/repodata/4c0ea0d0ca8fd81fd3a96cacabfbcf9e02c33125670505fcf20aacefab48df02-primary.sqlite.xz: [Errno 12] Timeout on ftp://ftp.cogeco.net/fedora/linux/releases/21/Everything/x86_64/os/repodata/4c0ea0d0ca8fd81fd3a96cacabfbcf9e02c33125670505fcf20aacefab48df02-primary.sqlite.xz: (28, '') Trying other mirror. Resolving Dependencies --> Running transaction check ---> Package docker-io.x86_64 0:1.5.0-1.fc21 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: docker-io x86_64 1.5.0-1.fc21 updates 5.8 M Transaction Summary ================================================================================ Install 1 Package Total download size: 5.8 M Installed size: 26 M Downloading packages: warning: /var/cache/yum/x86_64/21/updates/packages/docker-io-1.5.0-1.fc21.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 95a43f54: NOKEY Public key for docker-io-1.5.0-1.fc21.x86_64.rpm is not installed Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-21-x86_64 Importing GPG key 0x95A43F54: Userid : "Fedora (21) <fedora>" Fingerprint: 6596 b8fb abda 5227 a9c5 b59e 89ad 4e87 95a4 3f54 Package : fedora-repos-21-2.noarch (@anaconda) From : /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-21-x86_64 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : docker-io-1.5.0-1.fc21.x86_64 1/1 Verifying : docker-io-1.5.0-1.fc21.x86_64 1/1 Installed: docker-io.x86_64 0:1.5.0-1.fc21 Complete! time="2015-03-08T16:34:31-04:00" level="info" msg="+job serveapi(unix:///var/run/docker.sock)" time="2015-03-08T16:34:31-04:00" level="info" msg="Listening for HTTP on unix (/var/run/docker.sock)" time="2015-03-08T16:34:31-04:00" level="warning" msg="WARNING: Udev sync is not supported. This will lead to unexpected behavior, data loss and errors" time="2015-03-08T16:34:32-04:00" level="info" msg="+job init_networkdriver()" time="2015-03-08T16:34:32-04:00" level="info" msg="-job init_networkdriver() = OK (0)" time="2015-03-08T16:34:32-04:00" level="info" msg="WARNING: mountpoint for memory not found\n" time="2015-03-08T16:34:32-04:00" level="info" msg="Loading containers: start." time="2015-03-08T16:34:32-04:00" level="info" msg="Loading containers: done." time="2015-03-08T16:34:32-04:00" level="info" msg="docker daemon: 1.5.0 a8a31ef/1.5.0; execdriver: native-0.2; graphdriver: devicemapper" time="2015-03-08T16:34:32-04:00" level="info" msg="+job acceptconnections()" time="2015-03-08T16:34:32-04:00" level="info" msg="-job acceptconnections() = OK (0)" time="2015-03-08T16:34:36-04:00" level="info" msg="POST /v1.17/images/create?fromImage=fedora%3A21" time="2015-03-08T16:34:36-04:00" level="info" msg="+job pull(fedora, 21)" time="2015-03-08T16:34:36-04:00" level="info" msg="+job resolve_repository(fedora)" time="2015-03-08T16:34:36-04:00" level="info" msg="-job resolve_repository(fedora) = OK (0)" time="2015-03-08T16:34:36-04:00" level="info" msg="+job trust_update_base()" time="2015-03-08T16:34:36-04:00" level="info" msg="-job trust_update_base() = OK (0)" time="2015-03-08T16:34:37-04:00" level="info" msg="+job trust_key_check(/library/fedora)" time="2015-03-08T16:34:37-04:00" level="info" msg="-job trust_key_check(/library/fedora) = OK (0)" time="2015-03-08T16:34:37-04:00" level="info" msg="Image manifest for fedora:21 has been verified" 511136ea3c5a: Pulling fs layer 00a0c78eeb6d: Pulling fs layer 834629358fe2: Pulling fs layer 511136ea3c5a: Verifying Checksum 511136ea3c5a: Download complete 00a0c78eeb6d: Verifying Checksum 00a0c78eeb6d: Download complete 511136ea3c5a: Pull complete 00a0c78eeb6d: Pull complete 834629358fe2: Verifying Checksum 834629358fe2: Download complete 834629358fe2: Pull complete fedora:21: The image you are pulling has been verified. Important: image verification is a tech preview feature and should not be relied on to provide security. Status: Downloaded newer image for fedora:21 time="2015-03-08T16:38:48-04:00" level="info" msg="+job log(pull, fedora:21, )" time="2015-03-08T16:38:48-04:00" level="info" msg="-job log(pull, fedora:21, ) = OK (0)" time="2015-03-08T16:38:48-04:00" level="info" msg="-job pull(fedora, 21) = OK (0)" /bin/sh: line 7: killall: command not found virt-builder: error: docker -H unix:///var/run/docker.sock -d & sleep 5 docker pull fedora:21 killall docker : command exited with an error If reporting bugs, run virt-builder with debugging enabled and include the complete output: virt-builder -v -x [...] $ doh :P this is why hacking on these things takes so long... it did seem to work though. Once more with feeling (and killall): $ cat vb.sh #!/bin/bash virt-builder fedora-21 \ --install docker,psmisc \ --run-command ' docker -H unix:///var/run/docker.sock -d & sleep 5 docker pull fedora:21 killall docker ' $ ./vb.sh [ 2.0] Downloading: http://libguestfs.org/download/builder/fedora-21.xz [ 2.0] Planning how to build this image [ 2.0] Uncompressing [ 14.0] Opening the new disk [ 26.0] Setting a random seed [ 26.0] Installing packages: docker psmisc [ 60.0] Running: docker -H unix:///var/run/docker.sock -d & sleep 5 docker pull fedora:21 killall docker [ 316.0] Setting passwords virt-builder: Setting random password of root to hunter2 [ 316.0] Finishing off Output file: fedora-21.img Output size: 4.0G Output format: raw Total usable space: 5.2G Free space: 3.8G (72%) Which seemed to work :) So I might have a workaround, but there are definitely hacks involved. I see a few solutions: 1) Hopefully the docker people fix this daemon madness 2) If #1 is really impossible for some reason, perhaps virt-builder grows a --docker native command? The problem with this approach is that it might also want a rocket command, and others too. So perhaps something more general would be useful.
Yeah I suspect hard-coding docker daemons into virt-builder isn't going to work upstream. However suggestions for some kind of generic --daemon or --service option are welcome ... In general virt-builder's --run-command isn't that similar to the guest environment. Some things may be better done in --firstboot-command. See also: http://libguestfs.org/guestfs.3.html#running-commands
I've patched vagrant-builder to use this workaround: https://github.com/purpleidea/vagrant-builder/tree/feat/new-docker Two notes: 1) I added the --selinux-enabled flag 2) I've wholly not tested this at all (not even once) Figured I'd put this out here early. More permanent fixes either here and/or in docker are welcome, but probably not urgent at this time. Cheers, James
FWIW, I've pushed this to git master, and it seems to work :) w00t! Thanks Rich. I'd leave this open because it's kind of a hack, but if you prefer we close, that's fine too. Cheers!
This product has been discontinued or is no longer tracked in Red Hat Bugzilla.
Reopening because Virtualization Tools has not been discontinued.