Bug 1038092

Summary: %config(norepalce) files in jdk do not persist during update correctly
Product: Red Hat Enterprise Linux 7 Reporter: jiri vanek <jvanek>
Component: java-1.7.0-openjdkAssignee: jiri vanek <jvanek>
Status: CLOSED CURRENTRELEASE QA Contact: Lukáš Zachar <lzachar>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.0CC: dbhole, herrold, lzachar, omajid, ptisnovs, sochotni
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: java-1.7.0-openjdk-1.7.0.51-2.4.5.5.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1243947 (view as bug list) Environment:
Last Closed: 2014-06-13 11:16:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 1076552    
Bug Blocks: 1243947    
Attachments:
Description Flags
reproducer of behaviour
none
reproducers with similar structure as java and with aprox fix
jvanek: review?
Proposed scriptlet to address the issue
none
Proposed scriptlet to address the issue
none
Proposed scriptlet to address the issue
jvanek: review+
propsoed scripplet in lua
none
expanded script none

Description jiri vanek 2013-12-04 12:01:26 UTC
By installing the each release of jdk to new, unique directory, we created a bug which is hard to fix, and I'm not sure how serious it is.
When directory is same, and config files differs:
When rpm install by %config  dir/X file, it renames the old as dir/X.rpmsave and places new X - so program will automatically use the new config file
When rpm install by %config(noreplace)  dir/X file, it keeps old and save dir/X.rpmnew - so program will automatically use the old(with users precious changes)

When directory change  and config files differs:
When rpm install by %config  dir/X file, it keeps directory and renames the old as origdir/X.rpmsave and places new dir/X - so program will automatically use the new config file - thats correct (except nasty ghost dir)
When rpm install by %config(noreplace)  dir/X file, it do exactly as above, so norplace is practically ignored. Thats very wrong. Instead of old config the program will start usin new config. Also the old directory will never more removed.

There is 6 config(noreplace) files in openjdk, so we should probably handle it better then rpm warning message "original saved as origdir/X.rpmsave" 
%config(noreplace) %{_jvmdir}/%{jredir}/lib/security/US_export_policy.jar
%config(noreplace) %{_jvmdir}/%{jredir}/lib/security/local_policy.jar
%config(noreplace) %{_jvmdir}/%{jredir}/lib/security/java.policy
%config(noreplace) %{_jvmdir}/%{jredir}/lib/security/java.security
%config(noreplace) %{_jvmdir}/%{jredir}/lib/logging.properties
%config(noreplace) %{_jvmdir}/%{jredir}/lib/security/nss.cfg

from those especially java.security and logging.properties can be serious.

This was already confirmed as non rpm/yum bug, and the behaviour of rpm is same at el6 and el7.

Affected are fedora 19 and higher, rhel7, and all jdks7 - openjdk, ibm and oracle.

Comment 1 jiri vanek 2013-12-04 12:02:54 UTC
Created attachment 832597 [details]
reproducer of behaviour

When you install UjoBujo2, modify the config files, and then update to UjoBujo3, you will see the error.

Update from 1 to 2 is correct.

Comment 2 jiri vanek 2013-12-04 12:04:31 UTC
Omari in discussion suggested to save config files in non versioned directory:
eg  /etc/%{name}/, or ${jvmdir}/config/%{name} or wahtever... (where name stands for java-version-vendor, like java-1.7.0-openjdk)

The side effect of  this fix will be shared file between packages, when multiple (same) jdks are installed. Actually this is exactly one of issues we tried to fix with "muliple jdks" feature.

Comment 3 jiri vanek 2013-12-04 12:06:28 UTC
Tbh - I'm in favour to close this CLOSED -NOTABUG - ISAFEATURE, and solve this via solid documentation.

Comment 5 jiri vanek 2013-12-05 11:36:36 UTC
As devel ack was ++   -  note, coment #2  is *NOT* good solution. I'm in favour of not fixing it this way. I have just lack of ideas....

Comment 6 jiri vanek 2013-12-05 14:49:48 UTC
another idea is to manually copy old configs from previous jdk.
I do not like it neither.

Comment 7 jiri vanek 2013-12-06 10:34:32 UTC
Few more on "file copying"

This can work:
 - when there is *update*(not install!) ( $1 -eq 0 ?) , we can in  %pre[1] phase  copy *modified* (not untouched) (grep from rpm -qV ?*slow*) config(noreplace) files from previous, most recent (sort --version-sort ?)  previous installation. and save original one as .rpmnew. Tihs action should be printed to stdout

This is moreover replacing what rpm do (should do). 


[1]https://fedoraproject.org/wiki/Packaging:ScriptletSnippets#Scriptlet_Ordering

Comment 8 jiri vanek 2013-12-06 10:38:48 UTC
s/%pre/%post

sorry

Comment 9 jiri vanek 2013-12-06 11:24:57 UTC
s/( $1 -eq 0 ?)/( $1 -eq 2 ?)

Comment 10 jiri vanek 2013-12-06 16:18:20 UTC
Created attachment 833677 [details]
reproducers with similar structure as java and with aprox fix

I really have bad feelings about this scriplet stealing work of rpm. The behaviour of yum/rpm in some cases like undo transaction and o may be unpredictable. However belows should work


%post
#if update, check and copy modified config(norepalce)
if [ "x$1" = "x2" ] ; then
  filesWeCareOf="file1"
#this package isntalls to home
  fullPath=/home
#[0] is curent [1] is previous, somethink like "java-1.7.0-openjdk-\\([0-9].\\)\\+-\\([0-9].\\)\\+[a-z]\\+[0-9]\\+"  - jdks detectionm
  tmpfr=`mktemp`
  ls $fullPath/ | grep %{name}-[0-9]\\+-[0-9]\\+ | sort -Vr > $tmpfr
  currentDir=`sed -n  1p $tmpfr`
  previousDir=`sed -n  2p $tmpfr`
#get changed files in previous package
  for x in $filesWeCareOf ; do
#listign by name, both current and previous are listed
#so grep also dir
    rpm -qV  %{name} | grep  $previousDir/$x 1>/dev/null 2>/dev/null
    if [ $? = 0 ] ; then
      echo "warning: $fullPath/$previousDir/$x was modified, using. Saving $fullPath/$currentDir/$x as .rpmnew"
      mv $fullPath/$currentDir/$x $fullPath/$currentDir/$x.rpmnew
      cp $fullPath/$previousDir/$x $fullPath/$currentDir/$x
    fi
  done
fi

Comment 11 Deepak Bhole 2013-12-06 20:17:49 UTC
Created attachment 833746 [details]
Proposed scriptlet to address the issue

(In reply to jiri vanek from comment #10)
> Created attachment 833677 [details]
> reproducers with similar structure as java and with aprox fix
> 
> I really have bad feelings about this scriplet stealing work of rpm. The
> behaviour of yum/rpm in some cases like undo transaction and o may be
> unpredictable. However belows should work
> 

I don't think we should do anything in %post. Config files are best handled by RPM per its directives. The approach I had in mind was simply to emulate what rpm would find with a regular update, and then have it decide. For one, this means that we have to do what we want before a transaction begins. How about the attached patch (pasted below)?

I did very brief testing and it seems to work as expected. If there are any special corner cases you have, can you please try it with those?

%pretrans
# Check if this is an update
# In posttrans, $1 cannot be used
rpm -q %{name}
if [ "$?" = "0" ] ; then

  # Determine latest package with same name and its NVR
  LATEST_PKG=`rpm -q %{name} | sort | tail -n 1`
  LATEST_PKG_NVR=`rpm -q --queryformat="%%{name}-%%{version}-%%{release}.%{_arch}\n" $LATEST_PKG`

  # Copy over all config files, and let RPM deal with it
  for file in `rpm -qc $LATEST_PKG`; do

    # New location is just the newer NVR
    NEW_LOC=`echo $file | sed -e s:$LATEST_PKG_NVR:%{name}-%{version}-%{release}.%{_arch}:g`

    # Create the directory
    mkdir -p -m 777 `dirname $NEW_LOC`

    # Copy with -a to keep everything intact
    cp -a $file $NEW_LOC
  done
fi

Comment 12 Deepak Bhole 2013-12-06 20:21:16 UTC
Created attachment 833751 [details]
Proposed scriptlet to address the issue

Sorry, the above has a '-m 777' to mkdir that I had added as a test to ensure that permissions were not clobbered when the new files are layed out (they are not). Attached patch removes the '-m 777'.

Comment 13 jiri vanek 2013-12-09 12:48:44 UTC
Thank you for inovation!
It is really good idea to let rpm handle files. Form this point its much better.
Disadvantage is creating of destination directory(ies) which rpm should be handling.

few comments inline:

> %pretrans
> # Check if this is an update

according to http://fedoraproject.org/wiki/Packaging:ScriptletSnippets#Syntax the pretrasn should not be aware of uninstall, but if it works.. then the wiki page should be updated . And also in  uninstall it is NA, so you need 
[ x"$?" = x"0" ] ;  to avoid arg's "NPE" :)

> # In posttrans, $1 cannot be used
> rpm -q %{name}

> if [ "$?" = "0" ] ; then

>   # Determine latest package with same name and its NVR
>   LATEST_PKG=`rpm -q %{name} | sort | tail -n 1`
>   LATEST_PKG_NVR=`rpm -q --queryformat="%%{name}-%%{version}-%%{release}.%{_arch}\n" $LATEST_PKG`

Why this? You are using it only to get new directory name, and afaik the new directory is always same: jvmdir/uniquesuffix


>   # Copy over all config files, and let RPM deal with it
>   for file in `rpm -qc $LATEST_PKG`; do

rpm -qc $LATEST_PKG - thats cool, i was not aware of this !

>     # New location is just the newer NVR
>     NEW_LOC=`echo $file | sed -e s:$LATEST_PKG_NVR:%{name}-%{version}-%{release}.%{_arch}:g`

>     # Create the directory
>     mkdir -p `dirname $NEW_LOC`

>     # Copy with -a to keep everything intact
>     cp -a $file $NEW_LOC

I belive simple copy of file to  jvmdir/uniquesuffix/... should be enough
>   done
> fi

Comment 14 jiri vanek 2013-12-09 12:54:15 UTC
Also note - 

"This argument, accessed via $1 is the number of packages of this name which will be left on the system when the action completes" Here both me and you went wrong with test condition."

Here we both went wrong.

Comment 15 jiri vanek 2013-12-09 12:58:50 UTC
the " if [ "$?" = "0" ] ; then" issue can be avoided by  rpm -q to check if some other %{name} is installed.

Comment 16 Deepak Bhole 2013-12-09 15:43:26 UTC
(In reply to jiri vanek from comment #13)
> Thank you for inovation!
> It is really good idea to let rpm handle files. Form this point its much
> better.
> Disadvantage is creating of destination directory(ies) which rpm should be
> handling.
> 
> few comments inline:
> 
> > %pretrans
> > # Check if this is an update
> 
> according to
> http://fedoraproject.org/wiki/Packaging:ScriptletSnippets#Syntax the
> pretrasn should not be aware of uninstall, but if it works.. then the wiki
> page should be updated . And also in  uninstall it is NA, so you need 
> [ x"$?" = x"0" ] ;  to avoid arg's "NPE" :)
> 
> > # In posttrans, $1 cannot be used
> > rpm -q %{name}
> 
> > if [ "$?" = "0" ] ; then
> 

Right, pre-trans is not aware of $1, that is why I am doing rpm -q above. I am checking $? above, not $1. $? is the exit code of rpm -q

> >   # Determine latest package with same name and its NVR
> >   LATEST_PKG=`rpm -q %{name} | sort | tail -n 1`
> >   LATEST_PKG_NVR=`rpm -q --queryformat="%%{name}-%%{version}-%%{release}.%{_arch}\n" $LATEST_PKG`
> 
> Why this? You are using it only to get new directory name, and afaik the new
> directory is always same: jvmdir/uniquesuffix
> 

"Latest package" here is latest that is installed during pretrans (i.e. previous package).

> 
> >   # Copy over all config files, and let RPM deal with it
> >   for file in `rpm -qc $LATEST_PKG`; do
> 
> rpm -qc $LATEST_PKG - thats cool, i was not aware of this !
> 

Yeah, I didn't know until I looked at the man page either :)

> >     # New location is just the newer NVR
> >     NEW_LOC=`echo $file | sed -e s:$LATEST_PKG_NVR:%{name}-%{version}-%{release}.%{_arch}:g`
> 
> >     # Create the directory
> >     mkdir -p `dirname $NEW_LOC`
> 
> >     # Copy with -a to keep everything intact
> >     cp -a $file $NEW_LOC
> 
> I belive simple copy of file to  jvmdir/uniquesuffix/... should be enough

Are you saying we copy everything, not just the config files?

Comment 17 jiri vanek 2013-12-10 11:22:26 UTC
(In reply to Deepak Bhole from comment #16)
> (In reply to jiri vanek from comment #13)
> > Thank you for inovation!
> > It is really good idea to let rpm handle files. Form this point its much
> > better.
> > Disadvantage is creating of destination directory(ies) which rpm should be
> > handling.
> > 
> > few comments inline:
> > 
> > > %pretrans
> > > # Check if this is an update
> > 
> > according to
> > http://fedoraproject.org/wiki/Packaging:ScriptletSnippets#Syntax the
> > pretrasn should not be aware of uninstall, but if it works.. then the wiki
> > page should be updated . And also in  uninstall it is NA, so you need 
> > [ x"$?" = x"0" ] ;  to avoid arg's "NPE" :)
> > 
> > > # In posttrans, $1 cannot be used
> > > rpm -q %{name}
> > 
> > > if [ "$?" = "0" ] ; then
> > 
> 
> Right, pre-trans is not aware of $1, that is why I am doing rpm -q above. I
> am checking $? above, not $1. $? is the exit code of rpm -q

/me too dummy to live
Sorry - i really overlooked $? x $1, see comment 15 as proof I meant it well.

but still hidden issue here - in 

> 
> > >   # Determine latest package with same name and its NVR
> > >   LATEST_PKG=`rpm -q %{name} | sort | tail -n 1`
> > >   LATEST_PKG_NVR=`rpm -q --queryformat="%%{name}-%%{version}-%%{release}.%{_arch}\n" $LATEST_PKG`
> > 
> > Why this? You are using it only to get new directory name, and afaik the new
> > directory is always same: jvmdir/uniquesuffix
> > 
> 
> "Latest package" here is latest that is installed during pretrans (i.e.
> previous package).

nn - I meant why are you "LATEST_PKG_NVR=`rpm -q --queryformat="%%{name}-%%{version}-%%{release}.%{_arch}\n" $LATEST_PKG`" at all?

The LATEST_PKG should be all you need. Both Latest directory and latest id
> 
> > 
> > >   # Copy over all config files, and let RPM deal with it
> > >   for file in `rpm -qc $LATEST_PKG`; do
> > 
> > rpm -qc $LATEST_PKG - thats cool, i was not aware of this !
> > 
> 
> Yeah, I didn't know until I looked at the man page either :)
> 
> > >     # New location is just the newer NVR
> > >     NEW_LOC=`echo $file | sed -e s:$LATEST_PKG_NVR:%{name}-%{version}-%{release}.%{_arch}:g`
> > 
> > >     # Create the directory
> > >     mkdir -p `dirname $NEW_LOC`
> > 
> > >     # Copy with -a to keep everything intact
> > >     cp -a $file $NEW_LOC
> > 
> > I belive simple copy of file to  jvmdir/uniquesuffix/... should be enough
> 
> Are you saying we copy everything, not just the config files?

oh no!-)
I mean copy all as you do, just without the $LATEST_PKG_NVR liens. I will write small example a  bit later

Comment 18 jiri vanek 2013-12-10 11:57:17 UTC
> > > 
> > > > if [ "$?" = "0" ] ; then
> > > 
> > 
> > Right, pre-trans is not aware of $1, that is why I am doing rpm -q above. I
> > am checking $? above, not $1. $? is the exit code of rpm -q
> 
> /me too dummy to live
> Sorry - i really overlooked $? x $1, see comment 15 as proof I meant it well.
> 
> but still hidden issue here - in 
> 

in uninstall -  we dont wont to copy anything here (even not on itself)

Comment 19 jiri vanek 2013-12-10 13:18:13 UTC
#!/bin/sh

#%pretrans
#those tqo will be filled form macross
name=java-1.7.0-openjdk
uniquesuffix="java-1.7.0-openjdk-1.7.0.60-2.4.3.1.fc19.x86_64"
# Check if this is an update
# In posttrans, $1 cannot be used
#check if some %{name} is  installed
rpm -q $name > /dev/null
if [ "$?" = "0" ] ; then
  # if so, determine latest package with same name 
  # it si important to use version-sort, otherwise eg .9. will be wrongly sorted with .10.
  LATEST_PKG=`rpm -q $name | sort --version-sort | tail -n 1`
  CURRENT_PKG=$uniquesuffix

  # if current = latest, then it is uninstall and so nothing to do
    if [ "$CURRENT_PKG" != "$LATEST_PKG" ] ; then
    # Copy over all config files, and let RPM deal with it
    for file in `rpm -qc $LATEST_PKG`; do

      # New location is just the newer NVR
      NEW_LOC=`echo $file | sed -e s:$LATEST_PKG:$CURRENT_PKG:g`

      # Create the directory
      mkdir -p `dirname $NEW_LOC`

      # Copy with -a to keep everything intact
      cp -a $file $NEW_LOC
    done
  fi
fi

Comment 20 jiri vanek 2013-12-10 13:35:07 UTC
hmm...

 if [ "$CURRENT_PKG" != "$LATEST_PKG" ] ; then

is not enough when muliple jdks are installed and just one, particular, is going to be erased

Comment 21 jiri vanek 2013-12-10 13:42:59 UTC
perhaps:
...
INSTALLED_PKGS=`rpm -q $name | sort --version-sort`
INSTALL_UPDATE=true;
for file in $INSTALLED_PKGS; do
  if [ "$CURRENT_PKG" = "$file" ] ; then
    #already installed => reinstal, removal
    INSTALL_UPDATE=false
  fi
done

...
# if update/isntall
    if [ "$INSTALL_UPDATE" != "true" ] ; then
....

?

Comment 22 jiri vanek 2013-12-10 15:45:28 UTC
small - update - the scripts absed on Deepak's one are not working for noarch packages, but in jdk only javadoc (witn no config file) is no arch, so we can live with it.

Comment 23 Stanislav Ochotnicky 2013-12-10 17:09:29 UTC
Note that there are issues with the script:
 * %pretrans scriptlets can safely be written only in lua because they will likely fail during kickstart otherwise

 * running rpm inside rpm (rpm -qc call) is also unsupported by rpm upstream. Anything can happen.

Wouldn't creating a separate package that would be required by all JDKs and would contain these files be more appropriate? You can then use versioned conflicts/requires to limit what can be used where

Comment 24 jiri vanek 2013-12-10 17:18:17 UTC
The separate package have same drawbacks as config files saved in versionless directory and just linked frm jdk itself. 
The drawback is, that multiple installation of jdk of same origin  then share  the config files.

As multiple installations should help also with issue in config files (== swithced jdk via alternatives, also config is switched) then it can not be done.

Comment 25 Deepak Bhole 2013-12-10 20:08:57 UTC
I think we need to add more than than config file copy FWIW.

For example if someone has jars in in the ext directory, currently they stay with an upgrade, and with the new model they will not. Sounds like we may have to copy the entire old JDK :/

If we copy everything, we can do away with the rpm -qc call. The only RPM call left then is the rpm -q call to check of the older package is installed. If rpm calls are not reliable, the only way I think of is to see if old JDK dirs exist...

Comment 26 jiri vanek 2013-12-11 08:44:23 UTC
Yes, I have started to move to similar conclusion when trying the corner cases :(
also:
install (fresh, no jdk installed) - no copy
install as another jdk (older jdk detectted) -  copy
install as another jdk (newer jdk dtected ) -  copy?
install as another jdk (both older and newer jdk detected ) -  copy older?
update (older jdk detected) - *move* (most common use case)
   - when update is done and more jdks are installed, then we have to move from newest one, as ALL previous are erased
   - state when newer jdk is detected can not happen, as rpm will not update in such case
downgrade (newer jdk detected) - *move*
   - when downgrade is done and more jdks are installed, then we have to move from oldest one, as ALL previous are erased
   - downgrade from multiple installed was not yet tested:(
erase - no coping, moving
reinstall - same

note - it can happen that "previous" jdk is not fully removed during update or remove. Such an empty directory with file or two may confuse the directories detection form above. It is easily to be avoided, just have to be counted with.

another note - we have also files(in unique dirs) in /usr/lib/jvm-export - just links, and /usr/lib/jvm-private (just empty directories - http://pkgs.fedoraproject.org/cgit/java-1.7.0-openjdk.git/tree/java-1.7.0-openjdk.spec#n791 do they have still sense? )


Well pretrans do not have information about install, update, downgrade or erase, so I'm afraid we need to *copy* from newest (wrong when installing middle multiple jdk - it should copy from previous jdk) detected jdk always when current jdk is not detected:(

where "current jdk detected" is eg check if jvmdir/uniqueid/jr/bin/java exists, (and similar for older/newer in list sorted via --version-sort)

Comment 27 jiri vanek 2013-12-18 13:08:58 UTC
Interesting brainstroming may be patch to openjdk itself, so it will become to search a) (o b?) at its direcotry, and b) (or a?) at some shared directory.
Then the shared location can be in versionless directory, and still the configuration can be held unique for each jdk.

However the patch to jdk looks like too complex for this. And also this idea need to be "finished"

Comment 28 jiri vanek 2013-12-18 13:10:02 UTC
kernel like install-only solution.

No op here. It have hardcoded  yum/rpm exception.

Comment 29 jiri vanek 2013-12-18 13:13:29 UTC
As walking through this, the copying in pretrans looks like only reasonable approach.
However I'm strongly against copying whole jdk.

I'm for keeping the list of files (six config ones, and ext directory) which will be copied. Nothing more. 

I know this can lead to some bugs, but the changes in this list will be very rare.


According to anaconda guys, the kick-start should survive bash calls, but it will need some more testing. But we have to live without rpm command, so base everything just on sorted list of directories.

Comment 30 Deepak Bhole 2013-12-18 17:35:57 UTC
Created attachment 838485 [details]
Proposed scriptlet to address the issue

Doing so will mean that we have to statically encode the config file names. Right now the script uses an rpm -qc call which as you stated above, we cannot rely on in pretrans.

Please see attached new script.

Comment 31 jiri vanek 2013-12-19 08:57:19 UTC
just nits:
 *  sort --version-sort must be used, otherwise blah.10.blah x blah.1.blah will be sorted wrongly
 *  DEST=%{_jvmdir}/%{name}-%{version}-%{release}.%{_arch}/$file should be  DEST=%{_jvmdir}/%{uniquesuffix}/$file
 *  # Check if this is an update - is actually not true. It is just "check if some other openjdk of same vendor and version and arch is installed" . It trigger also for parallel install and downgrade. But still the case will be correct and  only comment should be updated

Otherwise yes, I think we can go ahead with this approach (with few more comments inside specfile, especially around %files sections)

Comment 32 jiri vanek 2014-01-24 15:38:15 UTC
https://bugzilla.redhat.com/show_bug.cgi?id=1055752#c1

 the /usr/lib/jvm/java-1.7.0-openjdk/lib/deployment.config (or more in lib?) should be handled by this script too....

Comment 34 Pavel Tisnovsky 2014-02-24 14:44:46 UTC
I just discovered that .jar files are saved into corresponding .rpmnew files even when the content is the same, but the files inside .jar archives have different date/time of creation. For example:

Archive:  /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.5.0.4.pre02.fc21.x86_64/jre/lib/security/US_export_policy.jar
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
       0  Defl:N        2   0% 01-31-2014 16:35 00000000  META-INF/
      96  Defl:N       91   5% 01-31-2014 16:35 382a1b18  META-INF/MANIFEST.MF
     141  Defl:N      115  18% 01-30-2014 14:34 7c5428a2  default_US_export.policy
--------          -------  ---                            -------
     237              208  12%                            3 files


Archive:  /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.5.0.4.pre02.fc21.x86_64/jre/lib/security/US_export_policy.jar.rpmnew
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
       0  Defl:N        2   0% 02-21-2014 12:27 00000000  META-INF/
      96  Defl:N       91   5% 02-21-2014 12:27 382a1b18  META-INF/MANIFEST.MF
     141  Defl:N      115  18% 02-20-2014 16:02 7c5428a2  default_US_export.policy
--------          -------  ---                            -------
     237              208  12%                            3 files

As you can see, content is really the same (CRC-32). I'm not sure if it's correct to create copies in that cases.

Comment 35 Pavel Tisnovsky 2014-02-24 14:57:03 UTC
Btw why the list of "changeable" files is too restricted?

There are many other config files that could be changed, for example:
sound.properties,
tz.properties
jre/lib/(amd64|...)/jvm.cfg
jre/lib/management/*

Comment 36 jiri vanek 2014-02-24 15:01:45 UTC
(In reply to Pavel Tisnovsky from comment #34)
> I just discovered that .jar files are saved into corresponding .rpmnew files
> even when the content is the same, but the files inside .jar archives have
> different date/time of creation. For example:
> 
> Archive: 
> /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.5.0.4.pre02.fc21.x86_64/jre/lib/
> security/US_export_policy.jar
>  Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
> --------  ------  ------- ---- ---------- ----- --------  ----
>        0  Defl:N        2   0% 01-31-2014 16:35 00000000  META-INF/
>       96  Defl:N       91   5% 01-31-2014 16:35 382a1b18 
> META-INF/MANIFEST.MF
>      141  Defl:N      115  18% 01-30-2014 14:34 7c5428a2 
> default_US_export.policy
> --------          -------  ---                            -------
>      237              208  12%                            3 files
> 
> 
> Archive: 
> /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.5.0.4.pre02.fc21.x86_64/jre/lib/
> security/US_export_policy.jar.rpmnew
>  Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
> --------  ------  ------- ---- ---------- ----- --------  ----
>        0  Defl:N        2   0% 02-21-2014 12:27 00000000  META-INF/
>       96  Defl:N       91   5% 02-21-2014 12:27 382a1b18 
> META-INF/MANIFEST.MF
>      141  Defl:N      115  18% 02-20-2014 16:02 7c5428a2 
> default_US_export.policy
> --------          -------  ---                            -------
>      237              208  12%                            3 files
> 
> As you can see, content is really the same (CRC-32). I'm not sure if it's
> correct to create copies in that cases.


I think that only solution for this is to mark them as config instead config(norepalce). Those files are regenerated each build.

Comment 37 jiri vanek 2014-02-24 15:14:23 UTC
ouch:
https://bugzilla.redhat.com/show_bug.cgi?id=1069224

Comment 38 jiri vanek 2014-03-03 15:21:55 UTC
Created attachment 869977 [details]
propsoed scripplet in lua

So this is the original script rewritten in lua.

Please note, it is using os.execute("cp".." -ar "..SOURCE.." "..DEST). I'm not sure if it will work. CP is to be in baseOS, but so is sh :(

Replacing "cp" by inner lua implementation may be tricky. If it will turn on that we can not use os.copy as above, then I would probably suggest to turn back to original fix, which is handling the config files manually (and so pretending yum behaviour - 833677 - reproducers with similar structure as java and with aprox fix)

Comment 39 jiri vanek 2014-03-12 09:24:18 UTC
Actually, I'm now thinking - will not the pretrasn script boke rpm -i "multiple install feature" ?

Comment 40 jiri vanek 2014-03-12 13:59:31 UTC
Created attachment 873571 [details]
expanded script

I have  pushed this to fedora today:
http://pkgs.fedoraproject.org/cgit/java-1.7.0-openjdk.git/commit/?id=bce3e606c023f2e320b7d03bea208ad5f9a36993
http://koji.fedoraproject.org/koji/taskinfo?taskID=6625870

Although it seems that this time it did not break anything. It seems that it do not work.

Atatched is the expanded script.

Comment 41 jiri vanek 2014-03-12 15:45:46 UTC
vercmp() 	Perform RPM version comparison (rpm >= 4.7.0) 	rpm.vercmp("1.2-1", "2.0-1") 

do exist as rpm lua extension. I will adapt the script somwhen (http://www.rpm.org/wiki/PackagerDocs/RpmLua)

Comment 42 jiri vanek 2014-03-12 18:10:00 UTC
 --trasnform substitute names to lua patterns
-local name = string.gsub(string.gsub(origname, "%-", "%%-"), "%.", "%%.")
-local javaver = string.gsub(origjavaver, "%.", "%%.")
---local arch = "x86_64"
-local arch ="%{_arch}"
+local name = string.gsub(string.gsub(origname, "", "%-"), "%.", "%.")
+local javaver = string.gsub(origjavaver, "%.", "%.")
+local arch ="x86_64"
 
 local jvms = { }
 
argh.. bloody rpm....

Comment 43 jiri vanek 2014-03-12 18:14:15 UTC
-  if (string.find(p, name..""..javaver..".*"..arch) ~= nil ) then
+  if (string.find(p, name.."%-"..javaver..".*"..arch) ~= nil ) then

I hope  qutro % an - will do the job :-/

Comment 44 Omair Majid 2014-03-12 21:03:03 UTC
(In reply to jiri vanek from comment #38)
> Please note, it is using os.execute("cp".." -ar "..SOURCE.." "..DEST). I'm
> not sure if it will work. CP is to be in baseOS, but so is sh :(

I think it's safe to assume that /bin/cp is missing too if /bin/bash is not present.

> Replacing "cp" by inner lua implementation may be tricky. If it will turn on
> that we can not use os.copy as above

Thinking about this, will cp not working be a real problem? In the case where /bin/cp is not around (new system/kickstart/compose) is a JVM likely to be around? I think as long as the script fails gracefully (and doesn't cause RPM to abort) we should be fine.

Comment 45 jiri vanek 2014-03-13 06:56:00 UTC
Its not true.
There is check, if some previous jvm is installed.  If some previous jvm is installed, then, and only after then, cp is used. If some jvm is instaled, then we can expect also base system installed.

In bash we can not do this. Because wee need the bash to check for installed jvm.

If no previous jvm is found, then cp is never called. And that is the cae of kick start.

Comment 46 jiri vanek 2014-03-13 11:25:59 UTC
So the updated version:
pushed 
http://pkgs.fedoraproject.org/cgit/java-1.7.0-openjdk.git/commit/java-1.7.0-openjdk.spec?id=eb474a9db60521fd8dcd91250857fc6508175898
and building http://koji.fedoraproject.org/koji/taskinfo?taskID=6628409

is working well (fixed this issue) in case of rpm -i, However during yum update, the files are not copied:-/

one more new issue:
Changed config files do remain in origial directory:
eg when updating   java-1.7.0-openjdk-1.7.0.60-2.4.5.1.fc20.x86_64, to java-1.7.0-openjdk-1.7.0.60-2.4.5.2.fc20.x86_64 in /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.5.1.fc20.x86_64 will remain (see .1. x .2.):

...
  Cleanup    : 1:java-1.7.0-openjdk-headless-1.7.0.60-2.4.5.1.fc20.x86_64                                                                                                                                                              12/12 
warning: /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.5.1.fc20.x86_64/jre/lib/logging.properties saved as /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.5.1.fc20.x86_64/jre/lib/logging.properties.rpmsave
...

Maybe we wont bash to  "rm  -rf" the old /usr/lib/jvm/oldjvm in postun?

Comment 47 jiri vanek 2014-03-13 15:30:51 UTC
%postun headless
  alternatives --remove java %{jrebindir}/java
  alternatives --remove jre_%{origin} %{_jvmdir}/%{jredir}
  alternatives --remove jre_%{javaver} %{_jvmdir}/%{jredir}
  alternatives --remove jre_%{javaver}_%{origin} %{_jvmdir}/%{jrelnk}

  # avoid unnecessary failure
  if [ -e %{_jvmdir}/%{uniquesuffix}  ] then 
    # as lua copied all necessary config files, we do not wont the double rpmnew and rpm.save
    rm -rf %{_jvmdir}/%{uniquesuffix}  
  fi
exit 0

will be new ppostun

Comment 48 jiri vanek 2014-03-13 16:31:24 UTC
just for record:

the script is working fine during yum update, the issue is  that RPM do not have information about  THIS file in db  it have information about file in OLD directory  so it simply overwrite the copied file.

Comment 54 jiri vanek 2014-04-02 17:22:32 UTC
As we discussed personally, the marking is no longer needed.

http://koji.fedoraproject.org/koji/taskinfo?taskID=6700189

build with returned rm ad usage of  OrderWithRequires java-1.7.0-headless...

Comment 57 jiri vanek 2014-04-11 09:15:44 UTC
There is one more issue - if yum reinstall or forced rpm -i of already installed version, then the lua script is actually copying into itself. Except few nasty error messages,there is no evil. I will fix this during CPU.

Comment 59 Ludek Smid 2014-06-13 11:16:47 UTC
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.

Comment 61 Omair Majid 2014-09-08 13:42:31 UTC
Removing the needinfo since this bug is already resolved.