Bug 1457912
| Summary: | /var/lib/docker mount does not persist across reboot. | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Dusty Mabe <dustymabe> | ||||
| Component: | docker-storage-setup | Assignee: | Vivek Goyal <vgoyal> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||
| Severity: | unspecified | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 26 | CC: | agrimm, dustymabe, vgoyal | ||||
| Target Milestone: | --- | Keywords: | Reopened | ||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2017-06-24 03:06:01 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
Can you provide container-storage-setup logs after reboot. After reboot are you running docker or not? This will be mounted when docker runs which in turn will run docker-storage-setup and then it will mount. I just tested on fedora 26 and it works for me. @vgoyal - looks like I needed one more step to my reproducer: Steps to Reproduce: 1. Boot an image from here: https://kojipkgs.fedoraproject.org/compose/branched/Fedora-26-20170531.n.0/compose/CloudImages/x86_64/images/ 2. Let the image boot. 2a. run a container ` docker run -it registry.fedoraproject.org:fedora:26 /bin/bash` 3. Perform reboot. 4. Observe /var/lib/docker is not mounted. Here are my logs for docker-storage-setup for 3 boots. 1st boot was initial boot. 2nd boot came up fine. On 2nd boot i created a container. On 3rd boot /var/lib/docker/ is not mounted. ``` [root@vanilla-f26atomic ~]# journalctl -u docker-storage-setup -- Logs begin at Thu 2017-06-01 17:41:34 UTC, end at Thu 2017-06-01 17:44:20 UTC. -- Jun 01 17:41:36 localhost.localdomain systemd[1]: Starting Docker Storage Setup... Jun 01 17:41:37 localhost.localdomain container-storage-setup[757]: CHANGED: partition=2 start=616448 old: size=83269632 end=83886080 new: size=85366784,end=85983232 Jun 01 17:41:37 localhost.localdomain container-storage-setup[757]: Physical volume "/dev/vda2" changed Jun 01 17:41:37 localhost.localdomain container-storage-setup[757]: 1 physical volume(s) resized / 0 physical volume(s) not resized Jun 01 17:41:37 localhost.localdomain container-storage-setup[757]: Logical volume "docker-root-lv" created. Jun 01 17:41:37 localhost.localdomain systemd[1]: Started Docker Storage Setup. -- Reboot -- Jun 01 17:42:15 vanilla-f26atomic systemd[1]: Starting Docker Storage Setup... Jun 01 17:42:15 vanilla-f26atomic container-storage-setup[727]: NOCHANGE: partition 2 is size 85366784. it cannot be grown Jun 01 17:42:15 vanilla-f26atomic container-storage-setup[727]: Physical volume "/dev/vda2" changed Jun 01 17:42:15 vanilla-f26atomic container-storage-setup[727]: 1 physical volume(s) resized / 0 physical volume(s) not resized Jun 01 17:42:15 vanilla-f26atomic systemd[1]: Started Docker Storage Setup. -- Reboot -- Jun 01 17:43:48 vanilla-f26atomic systemd[1]: Starting Docker Storage Setup... Jun 01 17:43:49 vanilla-f26atomic container-storage-setup[721]: NOCHANGE: partition 2 is size 85366784. it cannot be grown Jun 01 17:43:49 vanilla-f26atomic container-storage-setup[721]: Physical volume "/dev/vda2" changed Jun 01 17:43:49 vanilla-f26atomic container-storage-setup[721]: 1 physical volume(s) resized / 0 physical volume(s) not resized Jun 01 17:43:49 vanilla-f26atomic container-storage-setup[721]: mount: special device /dev/atomicos/docker-root-lv does not exist Jun 01 17:43:49 vanilla-f26atomic container-storage-setup[721]: ERROR: Failed to mount volume docker-root-lv on /var/lib/docker Jun 01 17:43:49 vanilla-f26atomic systemd[1]: docker-storage-setup.service: Main process exited, code=exited, status=1/FAILURE Jun 01 17:43:49 vanilla-f26atomic systemd[1]: Failed to start Docker Storage Setup. Jun 01 17:43:49 vanilla-f26atomic systemd[1]: docker-storage-setup.service: Unit entered failed state. Jun 01 17:43:49 vanilla-f26atomic systemd[1]: docker-storage-setup.service: Failed with result 'exit-code'. ``` Created attachment 1284484 [details]
verbose output
verbose `set -x` output was requested for when this happens on startup. Here is the output.
PR to fix this merged upstream. https://github.com/projectatomic/container-storage-setup/pull/245 thanks @vgoyal - let me know when this makes it into an RPM that we can test. docker-1.13.1-16.git51eb16e.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-4b5b8b0479 container-storage-setup-0.5.0-1.git9b77bcb.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-87e474598d docker-1.13.1-16.git51eb16e.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-4b5b8b0479 container-storage-setup-0.5.0-1.git9b77bcb.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-87e474598d docker-1.13.1-17.git51eb16e.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-84372bb930 container-storage-setup-0.5.0-1.git9b77bcb.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report. docker-1.13.1-17.git51eb16e.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-84372bb930 docker-1.13.1-18.git27e468e.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-740382b040 docker-1.13.1-18.git27e468e.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-740382b040 docker-1.13.1-19.git27e468e.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-36d05500ee docker-1.13.1-19.git27e468e.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-36d05500ee docker-1.13.1-19.git27e468e.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report. |
Description of problem: /var/lib/docker mount does not persist across reboot. On first boot: ``` [root@vanilla-f26atomic ~]# cat /etc/sysconfig/docker-storage-setup # Edit this file to override any configuration options specified in # /usr/share/container-storage-setup/container-storage-setup. # # For more details refer to "man container-storage-setup" STORAGE_DRIVER=overlay2 CONTAINER_ROOT_LV_NAME=docker-root-lv CONTAINER_ROOT_LV_MOUNT_PATH=/var/lib/docker GROWPART=true [root@vanilla-f26atomic ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT vda 252:0 0 41G 0 disk ├─vda1 252:1 0 300M 0 part /boot └─vda2 252:2 0 40.7G 0 part ├─atomicos-root 253:0 0 3G 0 lvm /sysroot └─atomicos-docker--root--lv 253:1 0 15.1G 0 lvm /var/lib/docker vdb 252:16 0 20G 0 disk [root@vanilla-f26atomic ~]# [root@vanilla-f26atomic ~]# [root@vanilla-f26atomic ~]# journalctl -u docker-storage-setup -- Logs begin at Thu 2017-06-01 13:43:34 UTC, end at Thu 2017-06-01 13:43:52 UTC. -- Jun 01 13:43:36 localhost.localdomain systemd[1]: Starting Docker Storage Setup... Jun 01 13:43:37 localhost.localdomain container-storage-setup[747]: CHANGED: partition=2 start=616448 old: size=83269632 end=83886080 new: size=85366784,end=85983232 Jun 01 13:43:37 localhost.localdomain container-storage-setup[747]: Physical volume "/dev/vda2" changed Jun 01 13:43:37 localhost.localdomain container-storage-setup[747]: 1 physical volume(s) resized / 0 physical volume(s) not resized Jun 01 13:43:37 localhost.localdomain container-storage-setup[747]: Logical volume "docker-root-lv" created. Jun 01 13:43:37 localhost.localdomain systemd[1]: Started Docker Storage Setup. [root@vanilla-f26atomic ~]# cat /etc/fstab # # /etc/fstab # Created by anaconda on Wed May 31 16:12:35 2017 # # Accessible filesystems, by reference, are maintained under '/dev/disk' # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info # /dev/mapper/atomicos-root / xfs defaults 0 0 UUID=e980060d-953c-41f7-a660-123676f6cfcd /boot ext4 defaults 1 2 ``` Version-Release number of selected component (if applicable): [root@vanilla-f26atomic ~]# rpm -q docker docker-1.13.1-7.git14cc629.fc26.x86_64 [root@vanilla-f26atomic ~]# rpm-ostree status State: idle Deployments: ● fedora-atomic:fedora/26/x86_64/atomic-host Version: 26.51 (2017-05-31 15:32:01) Commit: 770ae054136d69c69e81f8860be99290472371e20bed2abdce1c1a9468259935 This also happens on rawhide with docker-1.13.1-13.git51eb16e.fc27.x86_64. How reproducible: Always. Steps to Reproduce: 1. Boot an image from here: https://kojipkgs.fedoraproject.org/compose/branched/Fedora-26-20170531.n.0/compose/CloudImages/x86_64/images/ 2. Let the image boot. 3. Perform reboot. 4. Observe /var/lib/docker is not mounted.