Bug 1037634 - The devicemapper volumes are not removed after container is removed
Summary: The devicemapper volumes are not removed after container is removed
Alias: None
Product: Fedora
Classification: Fedora
Component: docker-io
Version: 20
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
Assignee: Lokesh Mandvekar
QA Contact: Fedora Extras Quality Assurance
Depends On:
TreeView+ depends on / blocked
Reported: 2013-12-03 14:10 UTC by Michal Fojtik
Modified: 2014-07-01 22:59 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2014-05-28 18:19:27 UTC
Type: Bug

Attachments (Terms of Use)

Description Michal Fojtik 2013-12-03 14:10:39 UTC
Description of problem:

After removing all containers in Docker, some of the devicemapper volumes are not removed and they are showed in 'df -h'

Version-Release number of selected component (if applicable):

[root@localhost postgresql]# docker -v
Docker version 0.7.0, build 0ff9bc1/0.7.0

How reproducible:

Create some containers and then use 'docker rm CONTAINER_ID' and remove them all. When the list of all containers (docker ps -a), is empty,
run 'df -h'. There are some devicemapper volumes from removed container still mounted in system.
After restarting docker service they seems to be properly removed.

Steps to Reproduce:
1. docker run ... (create some containers)
2. docker rm ALL_CONTAINERS
3. df -h (-> will show several devicemapper volumes mounted)
4. systemctl restart docker
5. df -h (-> all devicemapper volumes are erased)

Actual results:

[root@localhost postgresql]# docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
[root@localhost postgresql]#

[root@localhost postgresql]# df -h | grep mapper
/dev/mapper/fedora-root                                                                               9.2G  5.0G  3.7G  58% /
/dev/dm-3                                                                                             9.8G  390M  8.9G   5% /var/lib/docker/devicemapper/mnt/64e0619c473e974ac710568673fa0907fe9bc78f6998888ff41913cf6a69af0b
/dev/dm-4                                                                                             9.8G  388M  8.9G   5% /var/lib/docker/devicemapper/mnt/1bdd590de956281c7a5b1394cc0801ee879bc220be52e399eb61a9c3f2810c0d
/dev/dm-5                                                                                             9.8G  235M  9.0G   3% /var/lib/docker/devicemapper/mnt/97fc5bf7f8d42606fa896e1d391a0b882f78322ce0ff77c03fbd7f8e3b7a73ed
/dev/dm-6                                                                                             9.8G  388M  8.9G   5% /var/lib/docker/devicemapper/mnt/a567e6f3d26a5736d00a5e9be9a609b44ab13f0e43fc466f45beaa7ea9054766
/dev/mapper/docker-253:1-15108-7ee96a4f5167932745b68de0d026d750d12afa911a0e2591edd48efebdc8eb2f-init  9.8G  390M  8.9G   5% /var/lib/docker/devicemapper/mnt/7ee96a4f5167932745b68de0d026d750d12afa911a0e2591edd48efebdc8eb2f-init
/dev/mapper/docker-253:1-15108-c8c302f735e607c55ac4ae542fda9081959f128abbe3b126fa9afa541c84d52d-init  9.8G  518M  8.7G   6% /var/lib/docker/devicemapper/mnt/c8c302f735e607c55ac4ae542fda9081959f128abbe3b126fa9afa541c84d52d-init
/dev/mapper/docker-253:1-15108-5d0694e3a73324e395adfd03d7524c76f126c7449a4f5b73bac64b7bf26f51ef-init  9.8G  556M  8.7G   6% /var/lib/docker/devicemapper/mnt/5d0694e3a73324e395adfd03d7524c76f126c7449a4f5b73bac64b7bf26f51ef-init
/dev/mapper/docker-253:1-15108-f8a7e54667e09c1e9f3edf104124279d2e34fa1bade1ccbd8772a85369132cb6-init  9.8G  591M  8.7G   7% /var/lib/docker/devicemapper/mnt/f8a7e54667e09c1e9f3edf104124279d2e34fa1bade1ccbd8772a85369132cb6-init
/dev/mapper/docker-253:1-15108-fcf2d918ff47b9f24b1a9dcd6ea2829d4b20a5736f06d0a6d987c5cce74d4a55-init  9.8G  591M  8.7G   7% /var/lib/docker/devicemapper/mnt/fcf2d918ff47b9f24b1a9dcd6ea2829d4b20a5736f06d0a6d987c5cce74d4a55-init
/dev/mapper/docker-253:1-15108-004e09e25c19d765db7c75a767fffead530cac52aefd82d6acec5a9e765d4149-init  9.8G  591M  8.7G   7% /var/lib/docker/devicemapper/mnt/004e09e25c19d765db7c75a767fffead530cac52aefd82d6acec5a9e765d4149-init
/dev/mapper/docker-253:1-15108-5f32ced8736080d768cf72f210cda8ae06ef23d7018e5f07819c4c59607ae488-init  9.8G  591M  8.7G   7% /var/lib/docker/devicemapper/mnt/5f32ced8736080d768cf72f210cda8ae06ef23d7018e5f07819c4c59607ae488-init

Comment 1 Michal Fojtik 2013-12-03 14:11:41 UTC
[root@localhost postgresql]# docker images -a
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
mattdm/fedora       f19                 a567e6f3d26a        5 days ago          348.4 MB (virtual 348.4 MB)
mattdm/fedora       latest              64e0619c473e        6 days ago          349.8 MB (virtual 349.8 MB)
mattdm/fedora       f20                 1bdd590de956        5 weeks ago         348 MB (virtual 348 MB)
mattdm/fedora       f20rc3.small        97fc5bf7f8d4        10 weeks ago        198.4 MB (virtual 198.4 MB)

Comment 2 Alexander Larsson 2013-12-03 14:37:13 UTC
There are two issues here. 
First, there is a bug where we never remove the -init layers. 
And secondly, there are mounts for the images, which will go away if you remove them (rmi), but these mounts are kinda unnecessary as they are not currenly in use, and we should probably unmount them to keep the system "cleaner".

Comment 3 Michal Fojtik 2013-12-03 16:47:55 UTC
Also there might be additional bug, related to this. I just discovered that my /var/lib/docker folder use 11G of space, with no containers running and no extra images... Seems like they will need to be collected as well :-)

in /var/lib/docker:

11G     ./devicemapper
3.2G    ./devicemapper/mnt
11G     ./devicemapper

Comment 4 Alexander Larsson 2013-12-04 08:07:17 UTC
Michal: Not sure what you mean, you seem to have some mounts with 3.2 gigs of data in /var/lib/devicemapper/mnt, which implies that you have either a container or an image. Plus, the loopback mounts in the devicemapper folder are not zero size even if there are no containers/images in it.

Comment 5 Alexander Larsson 2013-12-06 11:42:39 UTC
The -init layer deletion is fixed in master and will be in 0.7.1

This will make us unmount not used devices: https://github.com/dotcloud/docker/pull/3073

Note You need to log in before you can comment on or make changes to this bug.