Bug 1204488 - report rpm errors to stderr and log file
Summary: report rpm errors to stderr and log file
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: 22
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: rpm-software-management
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1262424 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-03-22 15:41 UTC by Kashyap Chamarthy
Modified: 2019-03-04 04:00 UTC (History)
21 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-07-19 13:09:13 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Kashyap Chamarthy 2015-03-22 15:41:16 UTC
Problem description
-------------------

On Fedora 22, running:

  $ sudo dnf update -y
  [. . .]
  Running transaction check
  Transaction check succeeded.
  Running transaction test
  Transaction test succeeded.
  Running transaction
  Failed to obtain the transaction lock (logged in as: root).
  Error: Could not run transaction.


If not 'root', what the heck else am I supposed to run as?!?!


From /var/log/dnf.log:

  . . .
  Mar 22 11:29:43 CRITICAL Failed to obtain the transaction lock (logged in as: root).
  Mar 22 11:29:43 DDEBUG Cleaning up.
  Mar 22 11:29:44 SUBDEBUG 
  Traceback (most recent call last):
    File "/usr/lib/python2.7/site-packages/dnf/cli/main.py", line 84, in main
      return _main(base, args)
    File "/usr/lib/python2.7/site-packages/dnf/cli/main.py", line 141, in _main
      ret = resolving(cli, base)
    File "/usr/lib/python2.7/site-packages/dnf/cli/main.py", line 160, in resolving
      base.do_transaction()
    File "/usr/lib/python2.7/site-packages/dnf/cli/cli.py", line 217, in do_transaction
      super(BaseCli, self).do_transaction(display)
    File "/usr/lib/python2.7/site-packages/dnf/base.py", line 578, in do_transaction
      self._run_transaction(cb=cb)
    File "/usr/lib/python2.7/site-packages/dnf/base.py", line 674, in _run_transaction
      raise dnf.exceptions.Error(msg)
  Error: Could not run transaction.
  Mar 22 11:29:44 CRITICAL Error: Could not run transaction.
  . . .

Expected result
---------------

'dnf' should update/install the package as root!

Version
-------

  $ uname -r; rpm -q rpm dnf
  4.0.0-0.rc3.git0.1.fc23.x86_64
  rpm-4.12.0.1-7.fc22.x86_64
  dnf-0.6.4-2.fc22.noarch


Additional info
---------------

Attempt to download 'dnf-plugins-core' results in:

  $ dnf install dnf-plugins-core
  Using metadata from Sun Mar 22 11:15:08 2015
  Dependencies resolved.
  . . .
  Install  1 Package
  
  Total size: 71 k
  Installed size: 165 k
  Is this ok [y/N]: y 
  Downloading Packages:
  [SKIPPED] dnf-plugins-core-0.1.5-1.fc22.noarch.rpm: Already downloaded                                                                                                                       
  Running transaction check
  Transaction check succeeded.
  Running transaction test
  Transaction test succeeded.
  Running transaction
  Failed to obtain the transaction lock (logged in as: root).
  Error: Could not run transaction.


Several files in:

  $ ls -lsrt /var/lib/dnf/
  total 124
    4 drwxr-xr-x. 33 root root   4096 Sep  8  2014 yumdb
    0 -rw-r--r--.  1 root root      0 Mar 22 08:15 transaction-done.2015-03-22.08:15.52
  116 -rw-r--r--.  1 root root 118626 Mar 22 08:15 transaction-all.2015-03-22.08:15.52
    4 drwxr-xr-x.  3 root root   4096 Mar 22 11:29 history
  
  $ ls -lsrt /var/log/dnf.log
  4124 -rw-------. 1 root root 4215285 Mar 22 11:31 /var/log/dnf.log

Comment 1 Kashyap Chamarthy 2015-03-22 15:51:50 UTC
Note, there are no other 'dnf' or 'yum' (there's no 'yum' package itself on the system) processes running:
 
  $ ps -ef | egrep 'dnf|yum|rpm'


David Alan Gilbert on #fedora-qa suggested that something else may have the lock. . .

Also, I've read this slightly related bug: 

  https://bugzilla.redhat.com/show_bug.cgi?id=1111569

Comment 2 Honza Silhan 2015-04-03 16:01:55 UTC
Thanks for the report. Can you reproduce it anytime? DNF prints message about obtaining transition lock because it's the most probable scenario.

Florian, what else can be the cause of unsuccessful execution of transaction after calling rpm.TransactionSet.run(cb, "") returning empty list?

In case DNF get the empty list as result of `run()`:
-  if any item in transaction test `Failed()` => scriplet/non-fatal errors
-  else it's considered as transaction lock error

Comment 3 Kashyap Chamarthy 2015-04-07 12:44:50 UTC
(In reply to Jan Silhan from comment #2)
> Thanks for the report. Can you reproduce it anytime? DNF prints message
> about obtaining transition lock because it's the most probable scenario.

I'm afraid, I can't reproduce it consistently and don't have the machine handy now. If there's no actionable item here, I'm fine if you'd like to close it with as WORKSFORME.

> Florian, what else can be the cause of unsuccessful execution of transaction
> after calling rpm.TransactionSet.run(cb, "") returning empty list?
> 
> In case DNF get the empty list as result of `run()`:
> -  if any item in transaction test `Failed()` => scriplet/non-fatal errors
> -  else it's considered as transaction lock error

Comment 4 Florian Festi 2015-04-20 08:46:59 UTC
Well, the transaction lock is actually there to block root from other running transactions. But it looks like it doesn't get released in some cases. It should really be more robust and detect when it is stale. I'll keep this report open as a reminder to check the implementation.

Moving to rpm.

Comment 5 Panu Matilainen 2015-04-20 09:53:15 UTC
The transaction lock is an fcntl lock which the OS releases when the process terminates, so rpm fails to grab the lock then there will be one existing process holding it, stale transaction locks dont really exist.

I would look into the dnf code instead, it has some highly bogus-looking logic to try and guess whether rpm transaction failed because of locking failure or something else.

Comment 6 Christy Norman 2015-07-29 20:57:55 UTC
I've run into this today. Is there a way to get around it?

Comment 7 Jonathon Reinhart 2015-08-07 12:30:24 UTC
+1 I had recently used 'kill -9' to kill a 'sudo dnf install' that had hung hard for one reason or another.

Comment 8 Ľuboš Kardoš 2016-03-11 09:28:40 UTC
(In reply to Jan Silhan from comment #2)
> Thanks for the report. Can you reproduce it anytime? DNF prints message
> about obtaining transition lock because it's the most probable scenario.
> 
> Florian, what else can be the cause of unsuccessful execution of transaction
> after calling rpm.TransactionSet.run(cb, "") returning empty list?
> 
There are number of reasons why rpm transaction can fail. It is hard to say from information provided by dnf. It would be nice if dnf saved logs from rpm and if it didn't throw them away, see bug 1262424. That could help find out a root cause of a problem in cases like this.

Comment 9 Michal Luscon 2016-03-14 12:09:11 UTC
^^^^

Comment 10 Nicholas 2016-03-16 21:14:05 UTC
I've ran up against this issue today:

Fedora release 23 (Twenty Three)
VERSION="23 (Workstation Edition)"
4.4.4-301.fc23.x86_64

1.1.7
  Installed: dnf-0:1.1.7-2.fc23.noarch at 2016-03-14 15:08
  Built    : Fedora Project at 2016-03-09 16:45

  Installed: rpm-0:4.13.0-0.rc1.12.fc23.x86_64 at 2016-03-03 18:05
  Built    : Fedora Project at 2016-02-29 09:53



```
Mar 16 15:10:41 INFO --- logging initialized ---
Mar 16 15:10:41 DDEBUG timer: config: 6 ms
Mar 16 15:10:41 DEBUG cachedir: /var/cache/dnf
Mar 16 15:10:41 DEBUG Loaded plugins: config-manager, debuginfo-install, copr, reposync, Query, protected_packages, builddep, needs-restarting, download, generate_completion_cache, playground, noroot
Mar 16 15:10:41 DEBUG DNF version: 1.1.7
Mar 16 15:10:41 DDEBUG Command: dnf install clamav 
Mar 16 15:10:41 DDEBUG Installroot: /
Mar 16 15:10:41 DDEBUG Releasever: 23
Mar 16 15:10:41 DDEBUG Base command: install
Mar 16 15:10:41 DDEBUG Extra commands: ['clamav']
Mar 16 15:10:41 DEBUG repo: using cache for: fedora
Mar 16 15:10:41 DEBUG not found deltainfo for: Fedora 23 - x86_64
Mar 16 15:10:41 DEBUG not found updateinfo for: Fedora 23 - x86_64
Mar 16 15:10:41 DEBUG repo: using cache for: virtualbox
Mar 16 15:10:41 DEBUG not found deltainfo for: Fedora 23 - x86_64 - VirtualBox
Mar 16 15:10:41 DEBUG not found updateinfo for: Fedora 23 - x86_64 - VirtualBox
Mar 16 15:10:41 DEBUG repo: using cache for: slack
Mar 16 15:10:41 DEBUG not found deltainfo for: slack
Mar 16 15:10:41 DEBUG not found updateinfo for: slack
Mar 16 15:10:41 DEBUG repo: using cache for: google-chrome
Mar 16 15:10:41 DEBUG not found deltainfo for: google-chrome
Mar 16 15:10:41 DEBUG not found updateinfo for: google-chrome
Mar 16 15:10:41 DEBUG repo: using cache for: updates
Mar 16 15:10:41 DDEBUG timer: sack setup: 287 ms
Mar 16 15:10:41 DEBUG fedora: using metadata from Sat Oct 31 11:34:41 2015.
Mar 16 15:10:41 DEBUG virtualbox: using metadata from Fri Mar  4 12:38:45 2016.
Mar 16 15:10:41 DEBUG slack: using metadata from Mon Mar  7 13:50:51 2016.
Mar 16 15:10:41 DEBUG google-chrome: using metadata from Tue Mar 15 13:50:08 2016.
Mar 16 15:10:41 DEBUG updates: using metadata from Tue Mar 15 18:38:55 2016.
Mar 16 15:10:41 INFO Last metadata expiration check: 1:01:45 ago on Wed Mar 16 14:08:55 2016.
Mar 16 15:10:41 DEBUG --> Starting dependency resolution
Mar 16 15:10:42 DEBUG ---> Package clamav.x86_64 0.99-2.fc23 will be installed
Mar 16 15:10:42 DEBUG --> Finished dependency resolution
Mar 16 15:10:42 DDEBUG timer: depsolve: 670 ms
Mar 16 15:10:42 INFO Dependencies resolved.
Mar 16 15:10:42 INFO ==============================================================================================
 Package             Arch                Version                   Repository            Size
==============================================================================================
Installing:
 clamav              x86_64              0.99-2.fc23               updates              823 k

Transaction Summary
==============================================================================================
Install  1 Package

Mar 16 15:10:42 INFO Total size: 823 k
Mar 16 15:10:42 INFO Installed size: 2.2 M
Mar 16 15:10:44 INFO Downloading Packages:
Mar 16 15:10:44 SUBDEBUG Call: RPMPayload._end_cb: (<dnf.repo.RPMPayload object at 0x7f8731b234e0>, 1, 'Already downloaded'), {}
Mar 16 15:10:44 INFO Running transaction check
Mar 16 15:10:44 INFO Transaction check succeeded.
Mar 16 15:10:44 INFO Running transaction test
Mar 16 15:10:44 INFO Transaction test succeeded.
Mar 16 15:10:44 DDEBUG timer: transaction test: 82 ms
Mar 16 15:10:44 INFO Running transaction
Mar 16 15:10:45 DDEBUG RPM transaction start.
Mar 16 15:10:45 DDEBUG RPM transaction over.
Mar 16 15:10:45 CRITICAL Failed to obtain the transaction lock (logged in as: root).
Mar 16 15:10:45 DDEBUG Cleaning up.
Mar 16 15:10:45 INFO The downloaded packages were saved in cache until the next successful transaction.
Mar 16 15:10:45 INFO You can remove cached packages by executing 'dnf clean packages'.
Mar 16 15:10:45 SUBDEBUG 
Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/dnf/cli/main.py", line 60, in main
    return _main(base, args)
  File "/usr/lib/python3.4/site-packages/dnf/cli/main.py", line 120, in _main
    ret = resolving(cli, base)
  File "/usr/lib/python3.4/site-packages/dnf/cli/main.py", line 149, in resolving
    base.do_transaction(display=displays)
  File "/usr/lib/python3.4/site-packages/dnf/cli/cli.py", line 228, in do_transaction
    super(BaseCli, self).do_transaction(display)
  File "/usr/lib/python3.4/site-packages/dnf/base.py", line 615, in do_transaction
    self._run_transaction(cb=cb)
  File "/usr/lib/python3.4/site-packages/dnf/base.py", line 714, in _run_transaction
    raise dnf.exceptions.Error(msg)
dnf.exceptions.Error: Could not run transaction.
Mar 16 15:10:45 CRITICAL Error: Could not run transaction.
```

Comment 11 Honza Silhan 2016-03-21 12:48:38 UTC
*** Bug 1262424 has been marked as a duplicate of this bug. ***

Comment 12 Joe B 2016-06-28 18:07:29 UTC
ran into the same issue. have no way to troubleshoot.. debug log shows the same info as https://bugzilla.redhat.com/show_bug.cgi?id=1204488#c10 ... i will reboot and see if dnf recovers somehow unless there is some other recommended action.

Comment 13 Fedora Admin XMLRPC Client 2016-07-08 09:32:43 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 14 Fedora End Of Life 2016-07-19 13:09:13 UTC
Fedora 22 changed to end-of-life (EOL) status on 2016-07-19. Fedora 22 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 15 Ralph Loader 2019-01-28 07:37:23 UTC
This just happened to me.  At the moment I cannot do anything with dnf.

Presumably this will block me from installing critical security updates? (Fortunately I'm up-to-date within the last few hours).

Running `ps` I can't see any process holding stuff open; `rpm --rebuilddb` did not resolve the issue.

Comment 16 Ralph Loader 2019-01-28 07:58:19 UTC
Deleting both /var/cache/dnf/ and both /var/lib/dnf/ does not fix this.

Running strace on dnf, there are failures immediately before the error `:

openat(AT_FDCWD, "/etc/selinux/targeted/contexts/files/file_contexts.subs_dist", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/selinux/targeted/contexts/files/file_contexts.subs", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/etc/selinux/targeted/contexts/files/file_contexts", 0x7ffcbadef958) = -1 ENOENT (No such file or directory)
stat("/etc/selinux/targeted/contexts/files/file_contexts.bin", 0x7ffcbadef9f0) = -1 ENOENT (No such file or directory)

Is current dnf broken on non-selinux systems?

Comment 17 Marek Blaha 2019-01-30 07:36:31 UTC
Hi, please what version of dnf/libdnf are you using? (rpm -q dnf libdnf libsolv rpm)
Given that from year 2015 dnf code has changed *a lot*, it would be worth reporting a new bug. Anyway - I tried to run dnf on system  completly missing /etc/selinux directory without any problems. For fixing your issue we need more information - content of /var/log/dnf.log, what exactly error message is saying, ideally the way how to reproduce the bug.

Comment 18 Jens Petersen 2019-03-04 02:39:59 UTC
I just ran into this too... :-(
I think I did Ctrl-C earlier to dnf, sigh.

If there is a newer bug, please direct us there.

Comment 19 Jens Petersen 2019-03-04 04:00:40 UTC
I seem able to trigger this fairly easily with fedora-toolbox (with --release 30 at least).


Note You need to log in before you can comment on or make changes to this bug.