Bug 562773

Summary: yum history transaction begin fails in tr_TR.UTF-8 locale (returns NULL for transaction ID; causes traceback)
Product: [Fedora] Fedora Reporter: Sertaç Ö. Yıldız <sertacyildiz>
Component: pythonAssignee: Dave Malcolm <dmalcolm>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: low    
Version: 15CC: dmalcolm, ffesti, ivazqueznet, james.antill, jonathansteffan, maxamillion, ozan.caglayan, pmatilai, tim.lauridsen
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-08-07 20:12:31 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
/var/lib/yum/history/history-2010-02-03.sqlite none

Description Sertaç Ö. Yıldız 2010-02-08 11:05:38 UTC
Description of problem:
Traceback in yum.

Version-Release number of selected component (if applicable):
yum-3.2.25-1.fc12.noarch

How reproducible:
Occasional.

Steps to Reproduce:
$ yum update <some-package>
(or use PackageKit to update some packages)
  
Actual results:
Traacebacks below.

Additional info:
FWIW, I'm using tr_TR.UTF-8 locale.

Traceback (most recent call last):
  File "/usr/share/PackageKit/helpers/yum/yumBackend.py", line 2077, in _runYumTransaction
    rpmDisplay=rpmDisplay)
  File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 3959, in processTransaction
    self._doTransaction(callback,display=rpmDisplay)
  File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 4073, in _doTransaction
    self.runTransaction( cb=cb )
  File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 1070, in runTransaction
    self.history.beg(rpmdbv, using_pkgs, list(self.tsInfo))
  File "/usr/lib/python2.6/site-packages/yum/history.py", line 359, in beg
    self.trans_with_pid(pid)
  File "/usr/lib/python2.6/site-packages/yum/history.py", line 321, in trans_with_pid
    VALUES (?, ?)""", (self._tid, pid))
  File "/usr/lib/python2.6/site-packages/yum/sqlutils.py", line 168, in executeSQLQmark
    return cursor.execute(query, params)
IntegrityError: trans_with_pkgs.tid may not be NULL


Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in <module>
    yummain.user_main(sys.argv[1:], exit_code=True)
  File "/usr/share/yum-cli/yummain.py", line 315, in user_main
    errcode = main(args)
  File "/usr/share/yum-cli/yummain.py", line 263, in main
    return_code = base.doTransaction()
  File "/usr/share/yum-cli/cli.py", line 490, in doTransaction
    resultobject = self.runTransaction(cb=cb)
  File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 1070, in runTransaction
    self.history.beg(rpmdbv, using_pkgs, list(self.tsInfo))
  File "/usr/lib/python2.6/site-packages/yum/history.py", line 359, in beg
    self.trans_with_pid(pid)
  File "/usr/lib/python2.6/site-packages/yum/history.py", line 321, in trans_with_pid
    VALUES (?, ?)""", (self._tid, pid))
  File "/usr/lib/python2.6/site-packages/yum/sqlutils.py", line 168, in executeSQLQmark
    return cursor.execute(query, params)
sqlite3.IntegrityError: trans_with_pkgs.tid may not be NULL

Comment 1 James Antill 2010-02-08 13:49:50 UTC
Interesting.

Can you run: "ls -l /var/lib/yum/history"

I assume you aren't running out of disk space or anything?

If you need to work around it quick then running: "yum history new" which should make it work again (but if you have time I'd like to try and fix it).

Comment 2 Sertaç Ö. Yıldız 2010-02-08 14:16:32 UTC
(In reply to comment #1)
> Can you run: "ls -l /var/lib/yum/history"

$ LANG=C ls -l /var/lib/yum/history
total 628
-rw-r--r--. 1 root root 637952 Feb  8 13:14 history-2010-02-03.sqlite

> I assume you aren't running out of disk space or anything?

Currently I have over 2GB free space.

But to complete the preupgrade to F12 last week, I needed to free up some space on /. The date 2010-02-03 might refer to that.

Comment 3 James Antill 2010-02-08 19:47:08 UTC
Ok ... I don't see why sqlite/yum wouldn't be able to write to the DB.

Can you run:

yum list sqlite 'python-sqlite*' yum python

Would it be possible for you to upload the file: /var/lib/yum/history/history-2010-02-03.sqlite ?

Comment 4 Sertaç Ö. Yıldız 2010-02-08 22:10:47 UTC
$ yum list sqlite 'python-sqlite*' yum python
Loaded plugins: changelog, keys, presto, refresh-packagekit
Installed Packages
python.i686                                     2.6.2-4.fc12                                @anaconda-InstallationRepo-200911081854.i386
python-sqlite2.i686                             1:2.3.5-2.fc12                              @anaconda-InstallationRepo-200911081854.i386
sqlite.i686                                     3.6.20-1.fc12                               @anaconda-InstallationRepo-200911081854.i386
yum.noarch                                      3.2.25-1.fc12                               @anaconda-InstallationRepo-200911081854.i386

Comment 5 Sertaç Ö. Yıldız 2010-02-08 22:14:55 UTC
Created attachment 389633 [details]
/var/lib/yum/history/history-2010-02-03.sqlite

Comment 6 James Antill 2010-02-08 23:25:41 UTC
Looking at the history file, it says you did a transaction at 6am EST this morning ... installing texlive-luainputenc, with two failed transactions before that.

Was that the last transaction that worked?

Do you know what happened to the two previous transactions that failed?

What does the following say: rpm -V yum sqlite python

I can't see anything wrong with the history file, and I can create new entries in the trans_beg table here ... so if you run "yum history new", can you do an update?

Comment 7 Sertaç Ö. Yıldız 2010-02-09 00:11:07 UTC
(In reply to comment #6)
> Was that the last transaction that worked?

No.

> Do you know what happened to the two previous transactions that failed?

Probably the tracebacks I pasted were from those transactions. I think I'd updated directly with rpm, or with yum in C locale.

> What does the following say: rpm -V yum sqlite python

$ rpm -V yum sqlite python; echo $?
0

> I can't see anything wrong with the history file, and I can create new entries
> in the trans_beg table here ... so if you run "yum history new", can you do an
> update?    

The tracebacks happen occasionally, I can update without "yum history new".

I just updated gnome-mplayer without a traceback. But after that, "yum update texlive*" now results in the same traceback.

Comment 8 James Antill 2010-02-09 19:36:48 UTC
> The tracebacks happen occasionally, I can update without "yum history new".

 Interesting, do you know of any difference between the working and non-working runs? If not I guess the best thing to do is provide from debug code to see what is happening when it works/fails.

 It's weird because it's happening as both you and PK, which I'd have thought would be isolated from any weird environment stuff which might choose code in /usr/local or something.

Comment 9 James Antill 2010-02-10 01:42:17 UTC
After discussing this on IRC, it appears that yum it will randomly fail when run in the turkish locale, but never in the C locale. looking at:

http://svn.python.org/view/python/tags/r262/Modules/_sqlite/cursor.c?revision=71601&view=markup

...I see:

    while (isalpha(*src) && dst - buf < sizeof(buf) - 2) {
        *dst++ = tolower(*src++);
    }

...which looks really suspicious to me. 1) tolower() is a macro and so you can't do *src++ as an arg (probably ok). 2) isalpha()/tolower() in turkish locale is _not_ what you want here (almost certainly the bug IMO).
 Changing to the simple:

#define cisalpha(x) (((x) >= 'a' && (x) <= 'z') || ((x) >= 'A' && (x) <= 'Z'))
#define ctolower(x) ((x) - ('a' - 'A'))

...should dtrt.

Comment 10 James Antill 2010-02-10 01:56:52 UTC
And the above tolower() is wrong, as it needs to call cisupper() ... probably easiest to just copy them from glib ... Eg. http://library.gnome.org/devel/glib/stable/glib-String-Utility-Functions.html#g-ascii-tolower

Comment 11 Sertaç Ö. Yıldız 2010-03-14 17:53:48 UTC
still seeing the "sqlite3.IntegrityError" with python-2.6.2-7.fc12.i686

Comment 12 Bug Zapper 2010-11-03 22:44:30 UTC
This message is a reminder that Fedora 12 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 12.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '12'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 12's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 12 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 13 Sertaç Ö. Yıldız 2010-11-08 18:36:23 UTC
traceback still occurs with f12 (100% reproducible with texlive repo.)

Is this resolved in recent Fedora releases?

Comment 14 Bug Zapper 2010-12-03 23:08:54 UTC
Fedora 12 changed to end-of-life (EOL) status on 2010-12-02. Fedora 12 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 15 Sertaç Ö. Yıldız 2011-06-02 21:23:34 UTC
Still the same traceback with:
yum-3.4.1-3.fc15.noarch
python-2.7.1-7.fc15.i686

  File "/usr/bin/yum", line 29, in <module>
    yummain.user_main(sys.argv[1:], exit_code=True)
  File "/usr/share/yum-cli/yummain.py", line 275, in user_main
    errcode = main(args)
  File "/usr/share/yum-cli/yummain.py", line 212, in main
    return_code = base.doTransaction()
  File "/usr/share/yum-cli/cli.py", line 601, in doTransaction
    resultobject = self.runTransaction(cb=cb)
  File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 1465, in runTransaction
    self.skipped_packages, rpmdb_problems, cmdline)
  File "/usr/lib/python2.7/site-packages/yum/history.py", line 821, in beg
    self.trans_with_pid(pid)
  File "/usr/lib/python2.7/site-packages/yum/history.py", line 711, in trans_with_pid
    VALUES (?, ?)""", (self._tid, pid))
  File "/usr/lib/python2.7/site-packages/yum/sqlutils.py", line 168, in executeSQLQmark
    return cursor.execute(query, params)
sqlite3.IntegrityError: trans_with_pkgs.tid may not be NULL

Comment 16 Ozan Çağlayan 2012-03-24 13:17:55 UTC
Hi,

I can reproduce this error on an up-to-date F16 Verne by trying to install the adobe flash plugin from their sites.

I choose the YUM installation method, the repo rpm file gets downloaded, it is opened by software install. I type root's password and at the end of the operation I got the same traceback.

http://get.adobe.com/tr/flashplayer/completion/?installer=Flash_Player_11_for_other_Linux_%28YUM%29_64-bit

I'm on tr_TR.UTF-8 locale too. It's my 3rd Fedora 16 installation from the same usb stick. I added the adobe yum repository to all of them, no problem occurred on the other 2 installations which has en_US as locale, but on the tr_TR it happens every time.

I'll investigate further and try to post here if I found anything helpful. i->İ problem is a PITA, tolower() and toupper() shouldn't be used as they behave locale-aware. You should implement your own lower and upper functions as in the below bug report's provided patch to avoid weird problems and annoyances on tr_TR and other languages:

http://www.cups.org/str.php?L3800

Comment 17 Fedora End Of Life 2012-08-07 20:12:34 UTC
This message is a notice that Fedora 15 is now at end of life. Fedora
has stopped maintaining and issuing updates for Fedora 15. It is
Fedora's policy to close all bug reports from releases that are no
longer maintained. At this time, all open bugs with a Fedora 'version'
of '15' have been closed as WONTFIX.

(Please note: Our normal process is to give advanced warning of this
occurring, but we forgot to do that. A thousand apologies.)

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, feel free to reopen
this bug and simply change the 'version' to a later Fedora version.

Bug Reporter: Thank you for reporting this issue and we are sorry that
we were unable to fix it before Fedora 15 reached end of life. If you
would still like to see this bug fixed and are able to reproduce it
against a later version of Fedora, you are encouraged to click on
"Clone This Bug" (top right of this page) and open it against that
version of Fedora.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

The process we are following is described here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping