Bug 1169334 - docker run fails with " The database file is locked: database is locked"
Summary: docker run fails with " The database file is locked: database is locked"
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Fedora
Classification: Fedora
Component: docker-io
Version: 22
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Lokesh Mandvekar
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-12-01 11:01 UTC by Lokesh Mandvekar
Modified: 2015-03-06 17:59 UTC (History)
18 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-03-06 17:59:23 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Lokesh Mandvekar 2014-12-01 11:01:26 UTC
Description of problem:

$ sudo docker run -it fedora:rawhide bash
FATA[0000] Error response from daemon: The database file is locked: database is locked

Version-Release number of selected component (if applicable): docker-io-1.3.2-3.git353ff40.fc22.x86_64


How reproducible: while this may not be seen for the first few docker runs, it will pretty much show up by the 3rd-4th docker run


Steps to Reproduce:
1. install docker
2. run aforementioned command (try different image if need be)

Comment 1 Daniel Walsh 2014-12-01 13:11:20 UTC
https://github.com/docker/docker/issues/9428

Comment 2 Daniel Walsh 2014-12-01 13:12:28 UTC
If you delete the database and restart the daemon, it should clear up the problem.  Although I have no idea why this is happening.

Comment 3 Luwen Su 2014-12-03 07:08:19 UTC
(In reply to Lokesh Mandvekar from comment #0)
> Description of problem:
> 
> $ sudo docker run -it fedora:rawhide bash
> FATA[0000] Error response from daemon: The database file is locked: database
> is locked
> 
> Version-Release number of selected component (if applicable):
> docker-io-1.3.2-3.git353ff40.fc22.x86_64
> 
> 
> How reproducible: while this may not be seen for the first few docker runs,
> it will pretty much show up by the 3rd-4th docker run
> 
> 
> Steps to Reproduce:
> 1. install docker
> 2. run aforementioned command (try different image if need be)

Does this issue related kernel and systemd package?

I can't reproduce it on rhel7 but on fedora.
rhel7:
kernel-3.10.0-195.el7.x86_64
systemd-208-18.el7.x86_64
docker-1.3.2-3.el7.x86_64

fedora:
kernel-3.15.0-0.rc8.git4.1.fc21.x86_64
systemd-217-4.fc22.x86_64
docker-io-1.3.2-3.git353ff40.fc22.x86_64

Comment 4 Daniel Walsh 2014-12-03 09:28:07 UTC
I don't know, it does not make sense that it happens on one OS and not on the other.  It could be the version of sqlite that we are compiling against since this is where we are seeing the lock.



https://github.com/docker/docker/pull/9451

Seems to fix the problem on Rawhide.  And I am thinking we should ship this for all versions of docker-1.3.2  Just in case we are missing something on rhel7.

Comment 5 Colin Walters 2014-12-12 19:52:22 UTC
For me it reproduces every time with:

[root@localhost ~]# systemctl stop docker
Warning: Stopping docker.service, but it can still be activated by:
  docker.socket
[root@localhost ~]# rm -rf /var/lib/docker/ -rf
[root@localhost ~]# systemctl start docker
docker run b[root@localhost ~]# docker run busybox echo hello
Unable to find image 'busybox:latest' locally
busybox:latest: The image you are pulling has been verified
511136ea3c5a: Pull complete 
df7546f9f060: Pull complete 
e433a6c5b276: Pull complete 
e72ac664f4f0: Pull complete 
Status: Downloaded newer image for busybox:latest
hello
[root@localhost ~]# systemctl restart docker
[root@localhost ~]# docker run busybox echo hello
FATA[0000] Error response from daemon: The database file is locked: database is locked 
[root@localhost ~]#

Comment 6 Daniel Walsh 2014-12-13 12:40:43 UTC
Yes Docker has looked further at this and it seems to be a problem with their sqlite implementation interacting with golang-1.4.  I have asked Lokesh to block golang-1.4 from Fedora/Centos/RHEL until this is fixed.  Currently we are only seeing this breakage in Rawhide I believe.

rm -f /var/lib/docker/linkgraph.db; systemctl restart docker

Works for me. Although you might need to run this command again when the database gets locked up.

Comment 7 Colin Walters 2014-12-13 16:15:45 UTC
Fedora has no ability to block golang-1.4, we shipped a newer version and therefore must be better.

Comment 8 Lokesh Mandvekar 2014-12-14 04:31:07 UTC
(In reply to Daniel Walsh from comment #6)
> Yes Docker has looked further at this and it seems to be a problem with
> their sqlite implementation interacting with golang-1.4.  I have asked
> Lokesh to block golang-1.4 from Fedora/Centos/RHEL until this is fixed. 
> Currently we are only seeing this breakage in Rawhide I believe.
> 
> rm -f /var/lib/docker/linkgraph.db; systemctl restart docker
> 
> Works for me. Although you might need to run this command again when the
> database gets locked up.

(In reply to Colin Walters from comment #7)
> Fedora has no ability to block golang-1.4, we shipped a newer version and
> therefore must be better.

We can pretty much get around this by using the vendored golang (and other golang-* deps too) instead of the rpms, unless golang 1.4 is creating problems elsewhere too.

The latest rawhide build already uses vendored deps, but looks like the database locked problem still occurs :(

Comment 9 Daniel Walsh 2015-01-19 14:53:51 UTC
I am no longer seeing this with docker-1.4.1

Comment 10 M. Scherer 2015-01-25 00:14:09 UTC
Nope, it is not gone. I can reproduce on rawhide updated ( rebooted 2 days ago ), with vagrant:

 $ vagrant up --provider=docker                                                                                             
Bringing machine 'default' up with 'docker' provider...
==> default: Creating the container...
    default:   Name: ansible_foo_default_1422144721
    default:  Image: fedora:21
    default:    Cmd: id
    default: Volume: /home/misc/ansible_foo:/vagrant
A Docker command executed by Vagrant didn't complete successfully!
The command run along with the output from the command is shown
below.

Command: ["docker", "run", "--name", "ansible_foo_default_1422144721", "-d", "-v", "/home/misc/ansible_foo:/vagrant", "fedora:21", "id", {:notify=>[:stdout, :stderr]}]

Stderr: time="2015-01-25T01:12:01+01:00" level="fatal" msg="Error response from daemon: The database file is locked: database is locked" 


 $ cat Vagrantfile 
# -*- mode: ruby -*-
# vi: set ft=ruby :

# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.provider "docker" do |d|
    d.image = "fedora:21"
    d.cmd = ["id"]
  end
end

Comment 11 Jaroslav Reznik 2015-03-03 16:33:11 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 22 development cycle.
Changing version to '22'.

More information and reason for this action is here:
https://fedoraproject.org/wiki/Fedora_Program_Management/HouseKeeping/Fedora22

Comment 12 Colin Walters 2015-03-06 17:59:23 UTC
This one seems fixed now with 1.5.


Note You need to log in before you can comment on or make changes to this bug.