Bug 155730 - (2.6.11-1.14_FC3 sparse file?) x86_64 shadow-utils fc3-update install broken
Summary: (2.6.11-1.14_FC3 sparse file?) x86_64 shadow-utils fc3-update install broken
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: rpm
Version: 3
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Paul Nasrat
QA Contact: Mike McLean
URL:
Whiteboard:
: 155944 160125 (view as bug list)
Depends On:
Blocks: FC3Update
TreeView+ depends on / blocked
 
Reported: 2005-04-22 16:14 UTC by Jim Mauroff
Modified: 2007-11-30 22:11 UTC (History)
12 users (show)

Fixed In Version: 4.3.3-3.0.fc3.1
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2005-09-26 22:15:12 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
an output from 'rpm -Fvvv shadow-utils*' (7.29 KB, text/plain)
2005-05-06 20:32 UTC, Michal Jaegermann
no flags Details
results of 'strace rpm -Fvvv shadow-utils-4.0.3-56.x86_64.rpm' (374.09 KB, text/plain)
2005-05-06 20:34 UTC, Michal Jaegermann
no flags Details
sysrq showing tasks with stuck rpm (52.78 KB, text/plain)
2005-05-06 20:36 UTC, Michal Jaegermann
no flags Details

Description Jim Mauroff 2005-04-22 16:14:24 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; rv:1.7.3) Gecko/20041020 Firefox/0.10.1

Description of problem:
When trying to update shadow-utils-4.0.3-37 to shadow-utils-4.0.3-56 system eats up processing time and memory until the oom killer destroys pid.  This is only a problem on our x86_64 machines.  When using yum this appers to happen during the transaction tests.  Up2date's progress dialog states "Testing package set / solving RPM inter-dependencies."  An rpm install reports an out of memory condition if one has the patience to let it run for a looong while.  /var/log/messages reports an out of memory condition for all theree methods.

Version-Release number of selected component (if applicable):
4.0.3-56

How reproducible:
Always

Steps to Reproduce:
1.yum update shadow-utils
2.up2date
3.rmp -ivh shadow-utils-4.0.3-56.x86_64.rpm
  

Actual Results:  oom killer kills process, update not completed.

Expected Results:  Update performed.

Additional info:

Comment 1 Warren Togami 2005-04-23 10:18:23 UTC
http://forums.fedoraforum.org/showthread.php?t=51783
The package is somehow broken and nobody noticed since December?  Odd.  Similar
complaints in this thread.

Comment 2 Thorsten Leemhuis 2005-04-23 12:37:04 UTC
Odd, that's the right word. I looked into it, here are the things I found:

- I had a old FC3 x86_64 install that was keept up2date with yum. shadow-utils
was at 4.0.3-56 and everything was working fine. I installed several new
packages in the last days since installing 2.6.11-1.14_FC3 and everything worked
fine 

- After downgrading to the release package 4.0.3-38 I tried to reupdate to
4.0.3-56 -- the problem described in this bugreport showed up

- Killed rpm; Tried to update from 4.0.3-38 to 4.0.3-40 -- worked fine 

- Tried to update from 4.0.3-40 to 4.0.3-56 -- same problem as in this bug report

- Started with an older kernel (2.6.10-1.770_FC3) -- updateing shadow-utils to
4.0.3-56 works fine now

- Booted with 2.6.11-1.14_FC3 again and downgraded to 4.0.3-38; Tried updateing
to 4.0.3-56 while running strace. Last lines showed something with
/var/log/lastlog; Doing a $ ls -lh /var/log/lastlog showed
-r--------  1 root root 1,2T 23. Apr 13:28 /var/log/lastlog

Whats that? Search, search, ahh: Sparse file, See Bug 138676 and Bug 134803;
Okay, I still find that odd. Did a : > /var/log/lastlog and rebooted; Now it's
19M. Tried updateing to shadow-utils (still running with kernel 2.6.11-1.14_FC3)
to 4.0.3-56 and it worked fine now.

Looking closer this seems to be part of the problem:

$ rpm -qp shadow-utils-4.0.3-56.x86_64.rpm --scripts
postinstall scriptlet (using /bin/sh):
touch /var/log/lastlog
chmod 0400 /var/log/lastlog

shadow-utils-4.0.3-40.x86_64.rpm has no such %pre, seems the reason why that
update worked.

My current suspect: Maybe kernel 2.6.11-1.14_FC3 on x86_64 has problems with
sparse files?  

Comment 3 Peter Vrabec 2005-04-27 07:35:23 UTC
*** Bug 155944 has been marked as a duplicate of this bug. ***

Comment 4 Tim Waugh 2005-04-27 10:55:39 UTC
Why would this be a kernel problem?  It's the rpm process that's trying to
allocate 1068Gb: the kernel does its best to appease it.

Changing component to rpm.

Comment 5 Michal Jaegermann 2005-05-06 20:32:54 UTC
Created attachment 114094 [details]
an output from 'rpm -Fvvv shadow-utils*'

I bumped into the same problem.  It actually happens with various kernels
(2.6.11-1.14_FC3 and some rawhide kernels, both SMP and UP).  A weird thing
that in the meantime we run the same update on a number of x86_64 machines
and this is the first time I am aware the issue showed up.  At least some
of these had what appears a very similar software configuration to the box
in question so why this showed up now is a mystery.

Attaching strace to a stuck process does not show anything.  Not even
an unfished system call.

Attached are also results of 'SysRq-T' with 2.6.11-1.1284_FC4smp kernel
and results of running an update command under strace.	Terminating
'killed by SIGKILL' comes from 'pkill -9 rpm' as this is the only way to
terminate that process.

Comment 6 Michal Jaegermann 2005-05-06 20:34:58 UTC
Created attachment 114095 [details]
results of  'strace rpm -Fvvv shadow-utils-4.0.3-56.x86_64.rpm'

Comment 7 Thorsten Leemhuis 2005-05-06 20:36:20 UTC
/me wonders if a update from and fresh FC3 (not updated) to FC4t3/FC4 will fail
due to this error... I'll try to check next week

Comment 8 Michal Jaegermann 2005-05-06 20:36:31 UTC
Created attachment 114096 [details]
sysrq showing tasks with stuck rpm

Comment 9 Michal Jaegermann 2005-05-06 23:32:01 UTC
I forgot to add.  Booting with 'selinux=0' does not change anything.

Comment 10 Peter von der Ahe 2005-05-09 20:52:52 UTC
(In reply to comment #7)
> /me wonders if a update from and fresh FC3 (not updated) to FC4t3/FC4 will fail
> due to this error... I'll try to check next week

I have just installed a fresh FC3 (x86_64) and ran into this problem.

Comment 18 Bill Maidment 2005-06-02 13:31:52 UTC
I've just hit the same problem with 2.6.11-1.27_FC3. I installed a fresh FC3 and
then applied all the updates in small batches until I was left with perl and
shadow-utils. perl gave in to a --force but shadow-utils is buggered.

Comment 19 Robert Frank 2005-06-06 07:15:36 UTC
Someone else seems to have found the culprit: from speedyr6:

"Finally fixed by manually editing their Makefiles to point to the right mkinstalldirs that was in the root 
of the source tree. I can't figure out for the life of me what these folks were trying to do with the logic 
they were using to point to all of these different mkinstalldir files. It kept getting ../.././mkinstalldirs 
and I just had to manually point it to $srcdir/mkinstalldirs."

-Robert

Comment 20 Bill Maidment 2005-06-13 13:46:58 UTC
Can someone send me the revised version. I don't know how to amend an rpm. Thanks.

Comment 21 Jeff Johnson 2005-06-13 13:57:46 UTC
Detailed instructions on diagnosis and workarounds at bugzilla #160125.

Comment 22 Paul Nasrat 2005-06-14 16:36:57 UTC
A test rpm will be available from updates-testing:

rpm-4.3.3-3.0.fc3.

Comment 23 Axel Thimm 2005-06-18 10:41:19 UTC
Is FC4 (4.4.1-21) affected also? I found no mention of dealing with sparse files
or lastlog in either CHANGES or %changelog.


Comment 24 Paul Nasrat 2005-06-19 13:58:20 UTC
4.4.x has this patch.

Comment 25 Eric Goff 2005-06-24 22:31:49 UTC
You have a simple upgrade that will crash your FC3 system.
How is this not a high severity bug?

Comment 26 Jeff Johnson 2005-06-24 22:49:01 UTC
You have a simple solution for a reproducible problem. Why does severity matter?

Comment 27 Paul Nasrat 2005-06-28 20:11:55 UTC
re comment #25 - I'm still awaiting feedback from the testing update I pushed -
so far I have recieved none.  Positive feedback from testing will help expediate
the push to final updates.

yum --enablerepo=updates-testing update rpm

Comment 28 Marcel Ziswiler 2005-07-23 21:58:52 UTC
Nope, yum --enablerepo=updates-testing update rpm didn't do it. Still the same
hang at yum update or rpm -Uvh shadow-utils-4.0.3-56.x86_64.rpm. Looks like rpm
4.3.3-3.0.fc3.1 does not solve it.

Man, I thought I go crazy, brand new 64-bit server, new FC3 x86_64 installation
and doing the first set of updates crashes that thing!

Comment 29 Paul Nasrat 2005-07-25 15:40:11 UTC
rpm -q rpm
rpm -q rpm-libs

The fix really should work as it's what we are using in FC4 also.  Can you
confirm with strace where it is hanging using rpm -Uvvh.


Comment 30 Paul Nasrat 2005-08-16 15:03:53 UTC
*** Bug 160125 has been marked as a duplicate of this bug. ***

Comment 31 Paul Nasrat 2005-09-26 22:15:12 UTC
Since there are insufficient details provided in this report for us to
investigate the issue further, and we have not received the feedback we
requested, we will assume the problem was not reproduceable or has been fixed in
a later update for this product.  If you have further details, feel free to
reopen the report with the additional details attached.

Comment 32 Michal Szymanski 2005-12-08 22:03:25 UTC
Installing 'update-testing' rpm-4.3.3 does not help me, either.

BTW, syncing rpm and rpm-libs both to 4.3.3 breaks 'yum' completely :-(

>yum update
Traceback (most recent call last):
  File "/usr/bin/yum", line 6, in ?
    import yummain
  File "/usr/share/yum-cli/yummain.py", line 23, in ?
    import yum
  File "__init__.py", line 21, in ?
ImportError: /usr/lib64/librpm-4.3.so: undefined symbol: rpm_execcon

My setup is:
Fresh FC3 install on a dual-Opteron machine plus kernel pre-updated (with
dependecies) to latest 2.6.12-1.1381_FC3, yum pre-updated to latest 2.2.2-0.fc3.

Now, "yum update" goes all the way through downloading headers and packages and
hangs at Running Transaction Test

'top' shows following line:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 4153 root      25   0 1168g 3.5g 3.4g R 95.6 45.7   3:06.94 yum

I am not sure how the system allows a process to grow to 1168g on a 8GB memory +
16GB swap machine but maybe it is only the RESident 3.5g which matters.

Zeroing /var/log/lastlog helped but was described as "ugly workaround" so I
guess this bug is still unresolved.

Actually, it helped only in that the process did not hang. But it failed with
plenty of following messages:

file /..../.... from install of perl-5.8.5-20.FC3 conflicts with file from
package perl-5.8.5-9

I had to downgrade rpm-* to 4.3.2 (to make 'yum' work again), do 'yum remove
perl.i386' and retry.

regards, Michal.


Note You need to log in before you can comment on or make changes to this bug.