Bug 240697 - /usr/sbin/build-locale-archive: cannot open locale archive template
/usr/sbin/build-locale-archive: cannot open locale archive template
Status: CLOSED RAWHIDE
Product: Fedora
Classification: Fedora
Component: glibc (Show other bugs)
rawhide
All Linux
medium Severity medium
: ---
: ---
Assigned To: Jakub Jelinek
Brian Brock
:
Depends On:
Blocks: FC7Blocker
  Show dependency treegraph
 
Reported: 2007-05-20 12:50 EDT by Robert Scheck
Modified: 2008-08-02 19:40 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2007-05-29 10:39:44 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Robert Scheck 2007-05-20 12:50:52 EDT
Description of problem:
# rpm -Uvh i686/glibc-2.6-1.i686.rpm i386/glibc-devel-2.6-1.i386.rpm 
i386/glibc-headers-2.6-1.i386.rpm i386/glibc-common-2.6-1.i386.rpm
Vorbereiten...              ########################################### [100%]
   1:glibc-common           ########################################### [ 25%]
/usr/sbin/build-locale-archive: cannot open locale archive template 
file "/usr/lib/locale/locale-archive.tmpl": No such file or directory
Fehler: %post(glibc-common-2.6-1.i386) Skriptlet fehlgeschlagen, Beenden-
Status 1
   2:glibc                  ########################################### [ 50%]
   3:glibc-headers          ########################################### [ 75%]
   4:glibc-devel            ########################################### [100%]
# 

"Skriptlet fehlgeschlagen, Beenden-Status 1" means "Scriptlet failed, exit 
status 1".

# /usr/sbin/build-locale-archive
/usr/sbin/build-locale-archive: cannot open locale archive template 
file "/usr/lib/locale/locale-archive.tmpl": No such file or directory
# 

Version-Release number of selected component (if applicable):
glibc-2.6-1

How reproducible:
Just everytime for me, see above.

Actual results:
Error.

Expected results:
No error.
Comment 1 Robert Scheck 2007-05-20 12:56:43 EDT
Oh, I'm upgrading from glibc-2.5.90-21.
Comment 2 Jakub Jelinek 2007-05-21 11:42:37 EDT
Can't reproduce.  Can you try upgrading again and run the whole
command under strace -f ?
/usr/lib/locale/locale-archive.tmpl is one of the files in glibc-common
which rpm is supposed to unpack before the %post script is run.
The %post script then processes this file together with any preexisting
subdirectories in /usr/lib/locale/ as source and creates
/usr/lib/locale/locale-archive
file from it (and wipes the template).
Running /usr/sbin/build-locale-archive by hand is expected to fail, because
it is already missing, but rpm -Uvh shouldn't fail (and doesn't fail for me).
Comment 3 Robert Scheck 2007-05-21 17:33:16 EDT
It looks like something else is broken, because I'm no longer able to install 
older glibc having build-locale-archive too. Removing from F7 blocker list so 
far until I really know, what's up.
Comment 4 Jeff Johnson 2007-05-21 19:18:35 EDT
Hmmm, here's the immediate problem, /usr/lib/locale/locale-archive.tmpl not being installed
because
    
    D: /usr/lib/locale/locale-archive.tmpl skipped due to missingok flag
    ...
    D: fini      100644  1 (   0,   0)    65179424 /usr/lib/locale/locale-archive.tmpl;46522417 skip

Now to figger why the missingok flag is set in rpm-4.4.9 ...
Comment 5 Jeff Johnson 2007-05-21 19:28:50 EDT
And here's the next piece:

D: Conflicts: glibc-common <= 2.3.2-63                      NO  
==16368== 
==16368== Invalid read of size 1
==16368==    at 0x480635E: strcmp (mc_replace_strmem.c:341)
==16368==    by 0x482C5C7: checkPackageDeps (depends.c:1180)
==16368==    by 0x482C91A: checkPackageSet (depends.c:1308)
==16368==    by 0x482CD0E: rpmtsCheck (depends.c:1365)
==16368==    by 0x485EDDD: rpmInstall (rpminstall.c:659)
==16368==    by 0x4CD6: main (rpmqv.c:749)
==16368==  Address 0xCB86970 is 0 bytes inside a block of size 13 free'd
==16368==    at 0x48050FF: free (vg_replace_malloc.c:233)
==16368==    by 0x484E2EF: rpmdsNext (rpmlib.h:66)
==16368==    by 0x48489D5: rpmalAllSatisfiesDepend (rpmal.c:857)
==16368==    by 0x4848AF1: rpmalSatisfiesDepend (rpmal.c:891)
==16368==    by 0x482AD28: unsatisfiedDepend (depends.c:959)
==16368==    by 0x482C56F: checkPackageDeps (depends.c:1156)
==16368==    by 0x482C91A: checkPackageSet (depends.c:1308)
==16368==    by 0x482CD0E: rpmtsCheck (depends.c:1365)
==16368==    by 0x485EDDD: rpmInstall (rpminstall.c:659)
==16368==    by 0x4CD6: main (rpmqv.c:749)

Comment 6 Jeff Johnson 2007-05-21 21:03:07 EDT
The above problem is now fixed, nothing to do with this bug.

Here's the data in the pkg:
$ rpm -Kvv glibc-common-2.6-1.i386.rpm 
D: Expected size:     18628944 = lead(96)+sigs(180)+pad(4)+data(18628664)
D:   Actual size:     18628944
glibc-common-2.6-1.i386.rpm:
    Header SHA1 digest: OK (9ccf73fd3c6d70da0dec460f69ce5ab505a7ebb7)
    MD5 digest: OK (ff557d28ede0a5d5c4875f4a97b428f1)
$ rpm -qp --qf '[%{filenames} %{fileflags}\n]'  glibc-common-2.6-1.i386.rpm | grep tmpl
/usr/lib/locale/locale-archive.tmpl 9

So the flags are truly
    RPMFILE_CONFIG|RPMFILE_MISSINGOK
as can be seen in FC/devel/glibc/glibc.spec CVS:

1.301        (jakub    17-Apr-07): %attr(0644,root,root) %config(missingok) %{_prefix}/lib/locale/locale-
archive.tmpl

I have no idea what is being attempted, so its difficult to guess what should be done.

Lemme try a glibc build without missingok ...


Comment 7 Jeff Johnson 2007-05-21 21:20:24 EDT
FWIW, rpm-4.4.2 will behave identically, here's lib/rpmfi.c code in CVS:

2.44         (jbj      09-Jul-03):     if (lstat(fn, &sb)) {
2.44         (jbj      09-Jul-03):      /*
2.44         (jbj      09-Jul-03):       * The file doesn't exist on the disk. Create it unless the new
2.44         (jbj      09-Jul-03):       * package has marked it as missingok, or allfiles is requested.
2.44         (jbj      09-Jul-03):       */
2.44         (jbj      09-Jul-03):      if (skipMissing && (newFlags & RPMFILE_MISSINGOK)) {
2.44         (jbj      09-Jul-03):          rpmMessage(RPMMESS_DEBUG, _("%s skipped due to missingok flag
\n"),
2.44         (jbj      09-Jul-03):                      fn);
2.44         (jbj      09-Jul-03):          return FA_SKIP;
2.44         (jbj      09-Jul-03):      } else {
2.44         (jbj      09-Jul-03):          return FA_CREATE;
2.44         (jbj      09-Jul-03):      }
2.44         (jbj      09-Jul-03):     }

The difficulty in reproducing is likely due to whether the file exists (or not) ...
Comment 8 Jeff Johnson 2007-05-21 21:28:30 EDT
Note that rpm -Uvv --force has the behavior of erase-before-install which may also help explain why 
the reproducer is tricky.

All my comments are from examining rpm -Uvv --force glibc-common installs.

And I appear to have multiple glibc-common packages installed for whatever yum/rawhide/rpm reason:

$ rpm -q glibc-common
glibc-common-2.5.90-21.i386
glibc-common-2.5.90-22.i386
glibc-common-2.6-1.i386

 Let's clean up the mess ... same behavior after removing the 2 older glibc-common packages.
Comment 9 Jakub Jelinek 2007-05-22 03:07:51 EDT
So this works with the distro rpm (4.4.2) and doesn't with jbj's rpm?

The file really exist in the package, but has to be marked missingok,
because it is only used during %post and then unlinked.
locale-archive.tmpl together with /usr/lib/locale/*_*/ directories are sources
for build-locale-archive created locale-archive.  locale-archive.tmpl is
then unlinked because it is huge (~70MB) and not needed afterwards.
Comment 10 Jeff Johnson 2007-05-22 07:55:26 EDT
Too early to say Buggy!

Erasing a file contained in a package payload and marking config(missingok)
is unusual packaging that has not been attempted in any package I'm aware of
until now.
Comment 11 Jakub Jelinek 2007-05-22 08:11:41 EDT
Feel free to propose other packaging of this.
The needs:
1) the locale files are packed together, rpm really doesn't handle thousands of
files well
2) no duplication of the locale data files, when glibc-common is installed,
locales should only occupy roughly 70MB of disk space, not 140MB or more as they
used to before
3) it must be flexible with user supplied locales
4) programs should mmap locale-archive, not individual files in locale
subdirectories.

Until recently glibc-common contained almost 5000 files in ~ 1500 directories
and during %post created a ~ 70MB locale archive from it, so it ate 70+80MB of
diskspace, where the 80MB in the subdirectories were completely unused after
install (resp. glibc upgrades).
Comment 12 Jeff Johnson 2007-05-22 08:43:46 EDT
Re-installing w rpm-4.4.9 using --allfiles
    rpm -Uvv --force --allfiles glibc-common-2.6-1.i386.rpm
runs the %post scriptlet without errors.

I see no reason why rpm-4.4.2 should not behave identically to 4.4.9. (not looked).

Yes, it's a hard packaging problem. Lemme think a bit about adding a transient oneshot
install, like a self-erasing task sub-package.

No matter what glibc-common is tricky because no /bin/sh in chroot's.

FWIW, doing %post -p <lua> is likely easier to maintain than the current static helpers.
Comment 13 Jeff Johnson 2007-05-22 08:57:07 EDT
Again FYI: rpm has always had two glibc-common problems:
    1) partial hardlink sets to accomodate a locale install policy
    2) a busted algorithm (and combinatorial memory explosion) with
    lots of identical basenames like LC_MESSAGES.

1) appears to be a non-feature any more. I will start phasing out the mechanism in 4.4.9

2) was fixed on 4/1/2007 in 4.4.9 by adding a unique identifier to the join key to
prevent the combinatorially large memory explosion with lots of identical basenames.

With those flaws fixed, rpm performance should scale more linearally with lots of directories and files.
Comment 14 Robert Scheck 2007-05-23 17:52:06 EDT
Jakub, do you really consider this to be a FC7Blocker? PERSONALLY, I would 
revert the changes in glibc-common regarding build-local-archive and fix up
the 1) and 2) from comment #13 within Fedora RPM - just my 2 ct. Yes, I know 
we're 8 days before F7 goes gold. In fact, I'm able to reproduce this problem 
with Fedora RPM 4.4.2 e.g. by playing with --force but the reproducing seems 
not to be that easy like with RPM 4.4.9 - and I don't know why. You also could
only do the first thing of my proposal and fix the Fedora RPM for F8 (or even 
F7-post) to have better performing.
Comment 15 Jeff Johnson 2007-05-23 18:03:39 EDT
Try
    cp /dev/null /usr/lib/locale/locale-archive.tmpl
instead of removing. Then lose the tricky %config(missingok)
which was never intended for what you are trying to do.
Comment 16 Jakub Jelinek 2007-05-23 18:25:34 EDT
And replace with %verify(not md5 size mtime) ?
If that works, that would be great.  I was already considering bzip2ing
the locale-archive.tmpl and not removing it at all, but that means wasted 12MB
on the filesystem permanently.
Comment 17 Jakub Jelinek 2007-05-24 03:38:45 EDT
Summary: Test tempfile
Name: testtempfile
Version: 2.0
Release: 1
License: GPL
Group: Development/Libraries
BuildRoot: %{_tmppath}/%{name}-%{version}-buildroot
BuildArch: noarch
Prefix: /usr/opt/%{name}
%description
Other.
%prep
%build
%install
install -d -m 755 $RPM_BUILD_ROOT/usr/opt/%{name}
echo "%{name} %{version}" > $RPM_BUILD_ROOT/usr/opt/%{name}/%{name}
%clean
%post
if [ "$1" -ge 2 ]; then
    if echo "%{name} %{version}" | cmp -s - /usr/opt/%{name}/%{name}; then
        :
    else
        echo Failure
    fi
fi
> /usr/opt/%{name}/%{name} || :
%files
%defattr(0644,root,root,0755)
%dir /usr/opt/%{name}
%verify(not md5 size mtime)     /usr/opt/%{name}/%{name}

worked in my quick testing, can I rely on this?  If yes, I'll build new glibc.
Comment 18 Robert Scheck 2007-05-24 05:01:08 EDT
For me it worked in my testing, too. But as Jeff is the RPM god, I really would 
wait for a reply/short feedback by him.
Comment 19 Jeff Johnson 2007-05-24 07:59:48 EDT
You'll need to worry about empty chroo't's not having shell (I still suggest lua
so that others can see what glibc* helpers are doing), but basically yes, the
mechanism is simpler than %config(foo), and that means likelier to just "work".
Comment 20 Robert Scheck 2007-05-24 15:41:12 EDT
Jakub, Jesse - can we use LUA?
Comment 21 Will Woods 2007-05-29 10:28:39 EDT
Is the fix in comment #17 in glibc-2.6-3? If so, I'd like to close this bug.
Comment 22 Jakub Jelinek 2007-05-29 10:39:44 EDT
Re: comment #21: Yes, it is.
As for lua, it wouldn't help in any way to glibc-common's %post,
as merging of the locale archives with locale directories isn't something
I'd like to do in interpreted languages directly in the %spec file,
whether it is shell (using some statically linked shell and utilities), lua,
busybox or anything similar (and perl/python are neither desirable nor a good
idea to do it in).

Note You need to log in before you can comment on or make changes to this bug.