Description of problem: The service docker-storage-setup.service is failing. Version-Release number of selected component (if applicable): docker-1.13.1-22.gitb5e3294.fc26.x86_64 How reproducible: Start docker. Steps to Reproduce: 1. systemctl start docker 2. 3. Actual results: ● docker-storage-setup.service loaded failed failed Docker Storage Setup Expected results: The service should have started succesfully. Additional info: ● docker-storage-setup.service - Docker Storage Setup Loaded: loaded (/usr/lib/systemd/system/docker-storage-setup.service; disabled; vendor preset: disabled) Active: failed (Result: exit-code) since Wed 2017-11-01 12:26:29 EET; 4min 29s ago Process: 7074 ExecStart=/usr/bin/container-storage-setup (code=exited, status=1/FAILURE) Main PID: 7074 (code=exited, status=1/FAILURE) Νοέ 01 12:26:29 bb229 systemd[1]: Starting Docker Storage Setup... Νοέ 01 12:26:29 bb229 container-storage-setup[7074]: INFO: Volume group backing root filesystem could not be determined Νοέ 01 12:26:29 bb229 container-storage-setup[7074]: ERROR: No valid volume group found. Exiting. Νοέ 01 12:26:29 bb229 systemd[1]: docker-storage-setup.service: Main process exited, code=exited, status=1/FAILURE Νοέ 01 12:26:29 bb229 systemd[1]: Failed to start Docker Storage Setup. Νοέ 01 12:26:29 bb229 systemd[1]: docker-storage-setup.service: Unit entered failed state. Νοέ 01 12:26:29 bb229 systemd[1]: docker-storage-setup.service: Failed with result 'exit-code'.
Can you paste output of lsblk. docker-storage-setup will need a volume group (with free space) from where it can carve out a logical volume. There is no volume group. So you need to configure it right. You need to have an extra disk and pass that to docker-storage-setup. # cat /etc/sysconfig/docker-storage-setup DEVS=/dev/foo VG=docker-vg
The command output follows: $ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT loop0 7:0 0 100G 0 loop └─docker-8:1-677656-pool 253:0 0 100G 0 dm loop1 7:1 0 2G 0 loop └─docker-8:1-677656-pool 253:0 0 100G 0 dm sda 8:0 0 447.1G 0 disk ├─sda1 8:1 0 50G 0 part / ├─sda2 8:2 0 4G 0 part [SWAP] └─sda3 8:3 0 393.1G 0 part /home sdb 8:16 0 931.5G 0 disk └─sdb1 8:17 0 931.5G 0 part /opt sr0 11:0 1 1024M 0 rom If there is no volume group defined, it should have used the defaults for the root device, right? My /etc/sysconfig/docker-storage-setup is empty.
(In reply to Panos Kavalagios from comment #2) > > If there is no volume group defined, it should have used the defaults for > the root device, right? My /etc/sysconfig/docker-storage-setup is empty. Right but in your case there is no volume group backing rootfs. Root disk seems to be just partitioned and /dev/sda1 is root. That's why docker-storage-setup needs a disk and volume group to work with.
Adding a specific partition is not supported. Is there any other solution for systems without LVM?
Create a VG yourself and pass that VG to docker-storage-setup. # vgcreate docker-vg /dev/foo-part # cat /etc/docker-storage-setup VG=docker-vg
I think there's something else going on here. docker on Ff26 defaults to overlay2. The only way I can reproduce the original problem is by manually changing /etc/sysconfig/docker-storage-setup to add STORAGE_DRIVER=devicemapper WAG: is it possible that this system was originally installed as f25, and docker was installed but never used, and the system was upgraded to f26 without docker ever having been run?
The VG creation requires empty disk partitions of "Linux LVM" type to be added to the group, correct? So, in a system without LVM and already defined partitions is not possible (ok only with gparted probably). I confirm that docker was installed on F25 and without any custom configuration, but a docker image of MySQL was successfully tested.
Quick clarification: you do not have "a system without LVM". docker requires lvm2, so you have the lvm tools installed even if you have no lvm-managed partitions. But this is just a nit. I've been unable to reproduce this: starting with f25, installing and starting docker, 'docker info' shows thinpool as expected. Upgrading to f26, same thing (because /var/lib/docker was untouched). Stop docker, rm -rf /var/lib/docker, start docker, and now it's running overlay2. @Panos Kavalagios, can you think of any other factors that may be involved here?
LVM is also required by overlay2 storage driver? I don't have spare partition to create any LVM configuration and I would prefer overlayFS: # docker info Containers: 2 Running: 0 Paused: 0 Stopped: 2 Images: 1 Server Version: 1.13.1 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true Logging Driver: journald Cgroup Driver: systemd Plugins: Volume: local Network: bridge host macvlan null overlay Authorization: rhel-push-plugin Swarm: inactive Runtimes: oci runc Default Runtime: oci Init Binary: /usr/libexec/docker/docker-init-current containerd version: (expected: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1) runc version: N/A (expected: 9df8b306d01f59d3a8029be411de015b7304dd8f) init version: N/A (expected: 949e6facb77383876aeff8a6944dde66b3089574) Security Options: seccomp WARNING: You're not using the default seccomp profile Profile: /etc/docker/seccomp.json Kernel Version: 4.13.9-200.fc26.x86_64 Operating System: Fedora 26 (Twenty Six) OSType: linux Architecture: x86_64 Number of Docker Hooks: 3 CPUs: 4 Total Memory: 15.57 GiB Name: bb229 ID: UG5R:RA4A:IUE6:QQFO:32B3:GFQI:HQFL:IGD5:OWTV:65GQ:WU46:W7IM Docker Root Dir: /var/lib/docker Debug Mode (client): false Debug Mode (server): false Http Proxy: http://squid.eurodyn.com:8080/ No Proxy: localhost,127.0.0.1,.eurodyn.com Registry: https://registry.fedoraproject.org/v1/ Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false Registries: registry.fedoraproject.org (secure), registry.access.redhat.com (secure), docker.io (secure) How I can get rid of the docker-storage-setup.service startup failure with the above configuration without affecting docker normal operation?
You can specify to make use of overlay2 driver and then it should succeed. #cat /etc/sysconfig/docker-storage-setup STORAGE_DRIVER=overlay2
OK, thanks that worked. Now, if the default has changed to overlay2 in Fedora 26 from devicemapper, shouldn't STORAGE_DRIVER have overlay2 as the default option as well? Or at least docker of F26 to include /etc/sysconfig/docker-storage-setup file with that already set to overlay2.
That will happen if you install a fresh F26. But not over upgrade. Over upgrade user config files are supposed to remain same and new files are not supposed to replace existing files/settings.
I would propose something like the following in the RPM spec and %files section: %config(noreplace) /etc/sysconfig/docker-storage-setup so that the F26 could be notified for the new option in the .rpmnew version of the file. That would help to avoid configuration bugs like mine. I have performed no configuration in the docker sysconfig files in F25. I have only removed the /var/lib/docker/ contents to re-initialise the environment. I have also issued "dnf reinstall docker" in order to resolve the reported problem to no avail. Probably the "dnf remove docker" and then "dnf install docker" could have helped instead, but if there was a docker-storage-setup.rpmnew out there I would be fine. Just a simple change that would help the less power docker users after the migration to F26 to see that a specific systemd startup is failing.
yes this config should be set that way. %config(noreplace) /etc/sysconfig/docker-storage-setup
Franticek could you make sure that this is the setting?
Dan, Should "dnf remove docker" have removed /etc/sysconfig/docker-storage-setup, so that fresh installation of docker brings in new file? What about /etc/sysconfig/docker-storage?
Only if they are listed in payload and not modified. (If they are config(noreplace) I believe.
It is "%ghost %config(noreplace) %{_sysconfdir}/sysconfig/%{repo}-storage-setup" in f26 and later since february 2017. F25 lacks %ghost but is still %config(noreplace). docker-storage is also "%config(noreplace) %{_sysconfdir}/sysconfig/%{repo}-storage" in f26 and later.
Ok, I tested on F26, and removal of docker package does not remove /etc/sysconfig/docker-storage-setup. And that's a bug. %ghost means that file is not part of the package. If that's the case then specifying config(noreplace) after that does not make any sense. I suspect that could be part of the problem. If we just say. %ghost %{_sysconfdir}/sysconfig/%{repo}-storage-setup it might work. Remember, it needs to be done only if custom_storage needs to be set.
I removed config(noreplace) and tested and that fixed the problem. And now file is removed when docker package is removed. So we need following. # d-s-s specific %if %{custom_storage} %ghost %{_sysconfdir}/sysconfig/%{name}-storage-setup
Franticek or Lokesh can you make the change.
docker-1.13.1-43.git4402c09.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-c0b1fae074
It feels wrong to have a config file not marked as %config, but it seems to do what we want. Maybe we learn more at https://bugzilla.redhat.com/show_bug.cgi?id=1516938
docker-1.13.1-43.git4402c09.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-abfe613b7c
docker-1.13.1-43.git4402c09.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-c0b1fae074
docker-1.13.1-44.git584d391.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.
docker-1.13.1-44.git584d391.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.