Bug 1062959

Summary: Add fs sub-command, mainly for minimal docker container "creation"
Product: Red Hat Enterprise Linux 7 Reporter: James Antill <james.antill>
Component: yumAssignee: James Antill <james.antill>
Status: CLOSED CURRENTRELEASE QA Contact: Karel Srot <ksrot>
Severity: high Docs Contact:
Priority: medium    
Version: 7.0CC: dwalsh, james.antill, jzeleny, ksrot, lmiksik, riek
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: yum-3.4.3-116.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-06-13 11:36:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1062957    
Bug Blocks:    

Description James Antill 2014-02-09 09:12:20 UTC
Description of problem:

Add fs sub-command, and new yumdb entries, for easy manipulation of nodocs and _install_langs. Eg.

yum fs filter nodocs
yum fs filter langs en:es
yum fs refilter

...note that fixing bug#1062957 makes things a lot easier, but we can add workarounds (Eg. yum fs cleanup-langs).

Comment 1 Daniel Walsh 2014-02-10 00:40:27 UTC
Using my build example for installing httpd, would I execute those commands first?


# yum fs filter nodocs
# yum fs filter langs en
# yum fs refilter
# yum install .... httpd
# tar ... | docker

Comment 2 Karel Srot 2014-02-11 08:50:20 UTC
James,
could you please point me to some documentation of the new feature? From the examples above I do not quite follow what it is supposed to do. Thank you.

Comment 3 James Antill 2014-02-11 22:41:17 UTC
(In reply to Daniel Walsh from comment #1)
> Using my build example for installing httpd, would I execute those commands
> first?
> 
> 
> # yum fs filter nodocs
> # yum fs filter langs en

 Yeh, this is the setup. Telling yum to put tsflags=nodocs and _install_langs=en in all future transactions.

> # yum fs refilter

 This says to reinstall all packages which were installed using different nodocs/langs configuration than the current one.
 Note that bug#1062957 (and it's dep.) creates problems here, so I'd have to create some kind of "yum fs clean-filter" to remove the files on the FS that shouldn't be there. :(.

> # yum install .... httpd

 It would also apply to all future transaction though, so this would also have nodocs and langs=en.

> # tar ... | docker

Comment 4 James Antill 2014-02-11 22:41:56 UTC
(In reply to Karel Srot from comment #2)
> James,
> could you please point me to some documentation of the new feature? From the
> examples above I do not quite follow what it is supposed to do. Thank you.

 Sure, as soon as I write them :).

Comment 6 Daniel Walsh 2014-02-12 14:40:37 UTC
James, no way to get this to only effect one yum command.  IE I might want docs/lanquages on the host, but not in a chroot I am installing.

I guess you could always revert it.

Comment 7 James Antill 2014-02-12 16:09:13 UTC
 Yeh, you can use --setopt on the "yum install" you are doing for the container. EG.


 yum --setopt=override_install_langs=en --setopt=tsflags=nodocs --installroot=... install httpd

Comment 8 Daniel Walsh 2014-02-12 18:01:50 UTC
Awesome, when can I get this in Rawhide?  RHEL7?

Comment 9 James Antill 2014-02-13 00:18:07 UTC
Built for rawhide today, should be in it tomorrow ... also in the rawhide rebuild repos. Eg. http://copr-fe.cloud.fedoraproject.org/coprs/james/yum-rawhide/builds/


...Karel, I also did some docs in the man page.


       fs     This  command  has  a  few sub-commands to act on the filesystem
              data of the host, mainly  for  removing  languages/documentation
              for minimal installs:

               yum fs filters

               yum fs filter languages en:es

               yum fs filter documentation

               yum fs refilter [package(s)]

               yum fs refilter-cleanup [package(s)]

               yum fs du [path]

               yum fs status [path]

               yum fs diff [path]

              the first 3 being a simple interface to change yum.conf altering
              the  tsflags  and  override_install_langs  configurations.   The
              refilter  command is an optimized way of calling "yum reinstall"
              to reinstall the packages with  the  new  filters  applied.  The
              refilter-cleanup  command is needed because rpm doesn't actually
              remove the files on reinstall, as it  should.  And  the  du/sta‐
              tus/diff  commands  are included so you can easily see the space
              used/saved and any other changes.

Comment 12 Karel Srot 2014-03-04 08:59:05 UTC
FYI, I have reported bug 1062959.

James, could you please describe a bit more the 'diff' subcommand? I didn't identify what it is supposed to due. Thank you.

Comment 13 James Antill 2014-03-05 21:46:39 UTC
Probably easiest explanation is by example:

 yum fs diff /etc

...that's basically diff -ru on /etc vs. what is in the packages that are installed.

Comment 14 Karel Srot 2014-03-06 14:05:44 UTC
Thank you. I think I missed that information in the output between tons of warnings due to bug 1072258. :-)

Comment 15 Karel Srot 2014-03-06 14:17:09 UTC
James, are there any yum command that directly update yum.conf? I am not aware of any bug I do not such detailed insight and therefore I can't tell how such commands are supposed to work with yum.conf.

Why I am asking. I tried this:

# cp /etc/yum.conf /tmp
# yum -c /tmp/yum.conf fs filter docs
...
Enabling documentation filter.
fs filter done

# grep docs /tmp/yum.conf
# grep docs /etc/yum.conf
tsflags =nodocs

So the /etc/yum.conf was updated.. Funny thing starts when I try 

# yum -c /tmp/yum.conf fs filter nodocs
...
Already disabled documentation filter.
fs filter done

which is true for /tmp/yum.conf but previously yum was updating /etc/yum.conf.

So wouldn't it make sense to edit the config file passed via the -c option instead of /etc/yum.conf?

Comment 16 James Antill 2014-03-12 15:21:16 UTC
Yeh, not updating the -c version is a bit buggy ... the code was "borrowed" from yum-config-manager which does the same thing (ignores -c).

We could probably move to using .conf.config_file_path, but we'd need to tweak the surrounding code to work as well with installroot ... I think.

Comment 17 Karel Srot 2014-03-12 16:05:08 UTC
FYI, filed as bugs 1075706 and 1075708.

Comment 19 Karel Srot 2014-03-21 13:39:53 UTC
Hi James,
I am getting traceback after executing refilter-cleanup. yum is not able to delete a directory.

Running 'yum --disablerepo=\* --enablerepo=yum-fs-test -y fs refilter dummyPkgA dummyPkgB' (Expected 0, got 0)
Loaded plugins: auto-update-debuginfo, kabi, product-id, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
Loading support for Red Hat kernel ABI
Removing: /usr/share/doc/dummyPkgA-1.0
Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in <module>
    yummain.user_main(sys.argv[1:], exit_code=True)
  File "/usr/share/yum-cli/yummain.py", line 355, in user_main
    errcode = main(args)
  File "/usr/share/yum-cli/yummain.py", line 174, in main
    result, resultmsgs = base.doCommands()
  File "/usr/share/yum-cli/cli.py", line 571, in doCommands
    return self.yum_cli_commands[self.basecmd].doCommand(self, self.basecmd, self.extcmds)
  File "/usr/share/yum-cli/yumcommands.py", line 4885, in doCommand
    ret = self._fs_refilter_cleanup(base, extcmds)
  File "/usr/share/yum-cli/yumcommands.py", line 4724, in _fs_refilter_cleanup
    misc.unlink_f(fname)
  File "/usr/lib/python2.7/site-packages/yum/misc.py", line 953, in unlink_f
    os.unlink(filename)
OSError: [Errno 21] Is a directory: '/usr/share/doc/dummyPkgA-1.0'

Comment 20 James Antill 2014-03-25 04:08:42 UTC
Ok, I have a fix for this. Pushing to rawhide now, and push to el7 tomorrow.

Comment 23 Ludek Smid 2014-06-13 11:36:43 UTC
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.