Bug 1045494

Summary: yum post transaction action fails when package is removed
Product: Red Hat Enterprise Linux 6 Reporter: Robert Tomczyk <robert.x.tomczyk>
Component: yum-utilsAssignee: Packaging Maintenance Team <packaging-team-maint>
Status: CLOSED ERRATA QA Contact: Karel Srot <ksrot>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.4CC: akarlsso, james.antill, jzeleny, ksrot, robert.x.tomczyk, vmukhame
Target Milestone: rcKeywords: Patch
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: yum-utils-1.1.30-23.el6 Doc Type: Bug Fix
Doc Text:
Cause: enable post-transaction-action for directory which will be changed when some package is removed. Consequence: 'yum remove' crashes Fix: patch was added to fix the crash Result: 'yum remove' doesn't crash and the post-transaction-action is fulfilled successfully
Story Points: ---
Clone Of:
: 1127782 (view as bug list) Environment:
Last Closed: 2014-10-14 04:38:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Verified patch with fix for a bug.
none
Use transaction info to get package object -> txmbr.po none

Description Robert Tomczyk 2013-12-20 15:11:58 UTC
Description of problem:

Yum post transaction action was delivered to restart litpd service every time new python plugin is delivered and content of directory /opt/ericsson/nms/litp/lib is changed. 

The yum post action file is installed as: /etc/yum/post-actions/litp.action , below is the content of the file:
/opt/ericsson/nms/litp/lib:any:service litpd restart

All works fine during installation but when package is removed I'm getting the following error:

Installed size: 24 k
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Erasing    : ERIClitplibvirt_CXP9030547-1.0.44-1.noarch                                                                                                                  1/1 
Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in <module>
    yummain.user_main(sys.argv[1:], exit_code=True)
  File "/usr/share/yum-cli/yummain.py", line 285, in user_main
    errcode = main(args)
  File "/usr/share/yum-cli/yummain.py", line 219, in main
    return_code = base.doTransaction()
  File "/usr/share/yum-cli/cli.py", line 586, in doTransaction
    resultobject = self.runTransaction(cb=cb)
  File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 1590, in runTransaction
    self.plugins.run('posttrans')
  File "/usr/lib/python2.6/site-packages/yum/plugins.py", line 184, in run
    func(conduitcls(self, self.base, conf, **kwargs))
  File "/usr/lib/yum-plugins/post-transaction-actions.py", line 134, in posttrans_hook
    thispo = _get_installed_po(rpmdb, txmbr.pkgtup)
  File "/usr/lib/yum-plugins/post-transaction-actions.py", line 67, in _get_installed_po
    return rpmdb.searchNevra(name=n, arch=a, epoch=e, ver=v, rel=r)[0]
IndexError: list index out of range

I'm not sure why do we search for package in rmdb which we just removed.

Version-Release number of selected component (if applicable):

yum-plugin-post-transaction-actions-1.1.31-18.fc20.noarch

How reproducible:

Please try to remove a package for which post transaction action is enabled.

Steps to Reproduce:
1. Enable post-transaction-action for directory which will be changed when some package is removed.
2. Remove package in order to trigger yum-post-transaction-action

Actual results:

You should get "index out of range" exception as package is no longer in rpmdb. The post transaction action won't be executed but package will be removed.

Expected results:

The yum-post-transaction-action should be executed without any exception.

Additional info:

Please contact me for any more info if required.

Comment 2 Zdeněk Pavlas 2014-01-08 15:44:50 UTC
Upstream fix http://yum.baseurl.org/gitweb?p=yum-utils.git;a=commitdiff;h=069ccd1
The reporter confirms that a similiar patch seems to fix the problem.

Comment 3 Robert Tomczyk 2014-01-13 13:53:59 UTC
Hi  Zdeněk Pavlas 

 During removal of package the post transaction action didn't kick in at all
 I will include our fix for this.

Comment 4 Robert Tomczyk 2014-01-13 13:54:47 UTC
Created attachment 849373 [details]
Verified patch with fix for a bug.

Comment 5 Robert Tomczyk 2014-01-13 13:58:31 UTC
--- a/plugins/post-transaction-actions/post-transaction-actions.py
+++ b/plugins/post-transaction-actions/post-transaction-actions.py
@@ -129,10 +129,13 @@ def posttrans_hook(conduit):

             for txmbr in pkgset:
                 matched = False
+                
+                #print 'Package name: %s, output_state: %s' % (txmbr.name, txmbr.output_state)
                 if txmbr.output_state in TS_INSTALL_STATES:
                     thispo = _get_installed_po(rpmdb, txmbr.pkgtup)
                 else:
-                    continue
+                    # Removing, there is no info about package in rpmdb, using txmbr.po instead
+                    thispo = txmbr.po

                 if not yum.misc.re_glob(a_k):
                     if a_k in thispo.filelist + thispo.dirlist + thispo.ghostlist:
-- 
1.7.12.4

Comment 6 Robert Tomczyk 2014-01-13 14:52:59 UTC
Created attachment 849419 [details]
Use transaction info to get package object -> txmbr.po

As agreed with Zdenek Pavlas, we will take a package object info from current transaction.

Comment 7 Zdeněk Pavlas 2014-01-13 16:03:56 UTC
Slightly updated and merged. Thanks!

Comment 8 Zdeněk Pavlas 2014-01-14 11:38:48 UTC
Found out there is a problem with using txmbr.po unconditionally.  When installing/updating a package, txmbr.po.filelist access triggers download/decompress/query of the filelist metadata, if not already there.  Since we try to avoid using filelists, we should keep the rpmdb lookup code there.  Reverted the previous patches and made a new one to make backporting easier.

http://yum.baseurl.org/gitweb?p=yum-utils.git;a=commitdiff;h=d317fd0d8f56eb352c7dbb78bc283c208da9a561

Comment 11 Karel Srot 2014-04-28 15:59:46 UTC
Hello,
could you please summarize what is the expected behavior? I am a bit confused from the comments above because it seems that you want the transaction to be executed after the package removal... But according to the comments in the sample [1] file matches cannot be used with removes (and therefore neither for 'any')
What is the current upstream behavior?
Thank you.

[1]
$cat /usr/share/doc/yum-plugin-post-transaction-actions-*/sample.action
#action_key:transaction_state:command
# action_key can be: pkgglob, /path/to/file (wildcards allowed)
# transaction_state can be: install,update,remove,any
# command can be: any shell command
#  the following variables are allowed to be passed to any command:
#   $name - package name
#   $arch - package arch
#   $ver - package version
#   $rel - package release
#   $epoch - package epoch
#   $repoid - package repository id
#   $state - text string of state of the package in the transaction set
#
# file matches cannot be used with removes b/c we don't have the info available

Comment 12 Karel Srot 2014-05-02 08:24:10 UTC
James,
could you please take a look at #c11? Does it mean the the comment in sample.action is not valid anymore and the patch ensures we have all the data we need?

Comment 13 James Antill 2014-05-05 14:42:25 UTC
(In reply to Karel Srot from comment #11)
> Hello,
> could you please summarize what is the expected behavior? I am a bit
> confused from the comments above because it seems that you want the
> transaction to be executed after the package removal... But according to the
> comments in the sample [1] file matches cannot be used with removes (and
> therefore neither for 'any')

 Right, I wouldn't be shocked if this comment in the file was alluding to the behaviour in this bug ... in that it was known that the rpmdb lookup would fail.
 But it's possible it was written with the assumption that we'd call one of the yum APIs for mapping a file to a package (which wouldn't work against packages that no longer exist, from rpmdbs POV).

> What is the current upstream behavior?

 To quote Zdenek:

    I believe the code now works as origanally intended. When installing
    packages, txmbr.po is AvailablePackage instance and .filelist access
    may trigger filelist metadata download.
    
    Since this runs after the transaction, we may look the package up
    in rpmdb instead.  On removals, thispo is RPMInstalledPackage already.
    Thanks to Robert Tomczyk <robert.x.tomczyk> for reporting
    and testing this.

...so file matching should work for all cases.

(In reply to Karel Srot from comment #12)
> James,
> could you please take a look at #c11? Does it mean the the comment in
> sample.action is not valid anymore

 Yeh, it's probably worth fixing though (although you are somewhat unique in reading any documentation ;).

Comment 14 Karel Srot 2014-05-14 15:57:07 UTC
I did some testing of the fixed plugin and it almost seem to work. Below you can see what data were logged during various actions.

tst1234pkg* PACKAGE INSTALLATION
================================

install-action.log:
tst1234pkgB noarch 1.0 1 0 post-transaction-test-repo install
tst1234pkgA noarch 1.0 1 0 post-transaction-test-repo install


tst1234pkgA PACKAGE UPDATE
==========================

install-action.log:
tst1234pkgA noarch 1.0 2 0 post-transaction-test-repo updating

update-action.log:
tst1234pkgA noarch 1.0 2 0 post-transaction-test-repo updating

remove-action.log:
tst1234pkgA noarch 1.0 1 0 installed updated

tst1234pkg* PACKAGE REMOVAL
===========================

remove-action.log:
tst1234pkgA noarch 1.0 2 0 installed remove
tst1234pkgB noarch 1.0 1 0 installed remove


BUT, when I use filepath as an action_key only one package is logged.
E.g. suppose I have a package 
  tst1234pkgA containing /usr/local/tst1234pkgA
and package
  tst1234pkgB containing /usr/local/tst1234pkgB

When I configure action as follows:
/usr/local/tst1234pkg*:any:/bin/echo $name $arch $ver $rel $epoch $repoid $state
and remove both packages only one action would be executed (e.g. for package tst1234pkgB). The behaviour is same also for install action (and probably also update action). This is IMO wrong and both packages should be matched. 
Should I file another bug for that or can we fix it as a part of this bug report?

Comment 16 James Antill 2014-05-19 18:07:19 UTC
(In reply to Karel Srot from comment #14)
> Should I file another bug for that or can we fix it as a part of this bug
> report?

I would file that as a feature request ... as atm. we explicitly do a loop like:

for pkg in pkgs:
  matched = False
  # try to match files etc.
  if matched:
    break

...so it's not an accident, and while I can see that it might well be useful to keep trying to match (it might even be a better default for wildcards) I'll probably try and add it in a compat. way (Eg. having a different second item like update*/install*/remove*/any*).

Comment 17 Karel Srot 2014-05-20 05:49:27 UTC
(In reply to James Antill from comment #16)
> (In reply to Karel Srot from comment #14)
> > Should I file another bug for that or can we fix it as a part of this bug
> > report?
> 
> I would file that as a feature request ... as atm. we explicitly do a loop
> like:
> 
> for pkg in pkgs:
>   matched = False
>   # try to match files etc.
>   if matched:
>     break
> 
> ...so it's not an accident, and while I can see that it might well be useful
> to keep trying to match (it might even be a better default for wildcards)
> I'll probably try and add it in a compat. way (Eg. having a different second
> item like update*/install*/remove*/any*).

Filed as bug 1099338. I didn't file it as RFE because from my point of view current implementation is bogus. When I am matching files against the pattern I expect to get all files matching the pattern (and therefore execute the action for all respective packages), not just the first matching file.

Comment 23 errata-xmlrpc 2014-10-14 04:38:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-1411.html