Bug 1114422 - [rfe] Provide hooks functionality
Summary: [rfe] Provide hooks functionality
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: rawhide
Hardware: Unspecified
OS: Unspecified
high
unspecified
Target Milestone: ---
Assignee: Packaging Maintenance Team
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 1178969
TreeView+ depends on / blocked
 
Reported: 2014-06-30 04:35 UTC by Parag Nemade
Modified: 2016-02-22 22:08 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-02-22 22:08:51 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Bugzilla 1167881 None None None Never
Red Hat Bugzilla 1184808 None None None Never
Red Hat Bugzilla 1229946 None None None Never
Red Hat Bugzilla 1245198 None None None Never
Red Hat Bugzilla 1297058 None None None Never
Red Hat Bugzilla 1297520 None None None Never


Description Parag Nemade 2014-06-30 04:35:48 UTC
Description of problem:
I need similar functionality of slots and hooks like given on http://yum.baseurl.org/wiki/WritingYumPlugins

Sorry I could not find this required information in dnf (documentation). So, if I have missed it, maybe someone can point me. Else, can we implement this in dnf?

Version-Release number of selected component (if applicable):
nf-0.5.2-1.fc20.noarch
dnf-plugins-core-0.1.0-2.fc20.noarch


dnf-langpacks will need hooks

Comment 1 Tim Lauridsen 2014-06-30 04:44:16 UTC
dnf has the following hooks
http://akozumpl.github.io/dnf/api_plugins.html

I would be a good idea to tell what hooks you are missing and the detailed usecase for these hooks, so the dnf developers has something to work on.

Comment 2 Parag Nemade 2014-06-30 04:48:20 UTC
I need following yum hooks in dnf
================================================================================
config
    Called first as plugins are initialised. Plugins that need to extend Yum's
    configuration files or command line options should do so during this slot

postreposetup
    Called just after Yum initialises its repository information.

postresolve
    Called just after Yum finishes package resolution.

posttrans
    Called just after Yum has finished the RPM update transation.
===============================================================================
 

I see config is already implemented.

Comment 3 Tim Lauridsen 2014-06-30 05:07:34 UTC
posttrans = transaction in dnf

what do you need to do in each hooks, most actions can be moved into the hooks there is exists already in dnf

Comment 4 Parag Nemade 2014-06-30 05:17:30 UTC
the usecase for dnf-langpacks for these hooks are

postreposetup is needed to read the base packages marked as langpacks in comps file.

postresolve is needed for to find packages in a group install of language support groups. But maybe I can work without this as language groups have been removed since F17 I guess.

posttrans is needed to write the given language code to some file that will mark that language is installed on the system.

Comment 5 Parag Nemade 2014-06-30 05:21:55 UTC
Thanks Tim.

I will close this bug for now as I find that we have some hooks that I can use for dnf-langpacks. If I will find any issues, I will reopen this bug.

Comment 6 Parag Nemade 2014-11-26 10:22:29 UTC
How can I resolve again a resolved transaction? I am using resolved() hook and want to alter the package set and let the dnf run the transaction as usual.

I don't see a way to do this. Can someone point how can I do this? If not possible can we add a new hook for this in dnf?

Comment 7 Parag Nemade 2014-11-26 17:15:00 UTC
In short,
   We need a new hook in dnf where we can manipulate the packages that are going to be installed/removed. Maybe in between resolved() and transaction() hooks.

Long story,
When using dnf-langpacks plugin, if a user requests for any packages to be installed, we are reading the prepared transaction in resolved() hook. 

We find out if any conditional/langpack packages are selected for installation and also read the enabled languages. Then we find translation packages available for enabled languages for that langpack package and modify the transaction by adding such translation packages. 

Similar in opposite when user requests to remove langpack package then all its translation packages should get added in removal list of package in transaction.

e.g. if we have enabled language on the system are cs, de, es and ru. Then, if we run, it should behave like given below

$ sudo dnf install man-pages
Dependencies resolved.
================================================================================
 Package            Arch         Version                     Repository    Size
================================================================================
Installing:
 man-pages          noarch       3.69-2.fc21                 fedora       5.4 M
 man-pages-cs       noarch       0.18.20090209-18.fc21       fedora       567 k
 man-pages-de       noarch       1.8-1.fc21                  fedora       1.7 M
 man-pages-es       noarch       1.55-22.fc21                fedora       1.6 M
 man-pages-ru       noarch       3.41-4.20120901.fc21        fedora       1.6 M

Transaction Summary
================================================================================
Install  5 Packages

Total download size: 11 M
Installed size: 10 M
Is this ok [y/N]: y
Downloading Packages:
<snip>

similar in opposite when you just try to remove man-pages, it should remove all installed translation packages.

Comment 8 Parag Nemade 2015-01-06 03:11:01 UTC
I need to raise this bug's priority as now F22 anaconda development is also in need of this feature.

Comment 9 Jan Zeleny 2015-01-06 08:17:36 UTC
(In reply to Parag from comment #7)
> We find out if any conditional/langpack packages are selected for
> installation and also read the enabled languages. Then we find translation
> packages available for enabled languages for that langpack package and
> modify the transaction by adding such translation packages. 
> 
> Similar in opposite when user requests to remove langpack package then all
> its translation packages should get added in removal list of package in
> transaction.

I would just like to note that modifying transactions by plugins to achieve the behavior described above will be highly undesirable as of F22, as this exact use case is what rich dependencies in rpm are designed to address. Therefore, unless there is another use case, I would be very careful with the hook that enables plugins to modify transactions after they are resolved.

Comment 10 Jens Petersen 2015-01-13 05:53:06 UTC
http://www.rpm.org/wiki/PackagerDocs/Dependencies#Weakdependencies

Could dnf-langpacks somehow promote Enhances to Supplements
for the langpacks of a specific locale?

Jan, or is there a better approach to solve this?

Comment 11 Jan Zeleny 2015-01-13 12:34:37 UTC
I'm not sure. One possibility would be to implement the requested hooks but in that case I would expect the depsolving step to be repeated if the transaction was modified - that would ensure the transaction was not broken by a plugin but it can also have some performance implications which I don't like much.

I am not sure who of the interested people will be visiting DevConf in February but we plan to have a session focused on rich dependencies there. We would like to use this session to lay down some guidelines how to work with them and come up with some examples how to use them correctly. This would be a great opportunity to solve this use case - that way the langpacks in F22+ could be based solely on Supplements and "AND" operator.

Example (man-pages-cs):
Supplements: man-pages AND langpack-cs

Does this answer your question?

Comment 12 Parag Nemade 2015-02-05 16:07:37 UTC
Adding further information. Specifying more that one supplements be on the same line or different line is not working currently.

Comment 13 Parag Nemade 2015-06-03 14:59:44 UTC
Now that we have F22 released and F23 just began, Can this request be considered again? Note we are missing the anaconda installation with langpacks because of this bug.

Also, above suggested approach does not look good as I need to change many many packages in Fedora by adding Supplements. 

I am not sure if the requested hook can be added as a plugin for now. Can someone please help completing dnf-langpacks 100% as compared with yum-langpacks? I really don't want to miss F23 now.

Comment 14 Parag Nemade 2015-06-16 08:41:48 UTC
Any thoughts, as I already said its not possible to update many langpacks packages and I have not got success in using AND and OR operations. I see the easiest (yum compatible) way is to add new hook that will modify transaction and re-resolve packages and proceed with installation.

Comment 15 Honza Silhan 2015-07-14 14:51:33 UTC
(In reply to Parag from comment #13)
> Now that we have F22 released and F23 just began, Can this request be
> considered again? Note we are missing the anaconda installation with
> langpacks because of this bug.

If the only Anaconda is the case then it could find all lang packages and install them in the next transaction.

I still have a concerns about adding a hook after resolution. When you add them into transaction you can get broken dependencies. If you do another resolution after new packages are added then you can't be 100% sure you get the superset. I am inclined to clean solution of taking advantage of rich dependencies.

Pavel, do you know in which state are / will be rich dependencies in F23?

Parag, could the langpack plugin install packages in the next DNF run or when "dnf langpacks" is executed?

Comment 16 Parag Nemade 2015-07-19 07:18:53 UTC
Hi Jan,
  last few days I worked on completing config file part (langpacks.conf) in dnf-langpacks and now anaconda developers also supported missing functionality in anaconda -> https://github.com/rhinstaller/anaconda/pull/222/commits

No. dnf-langpacks plugin cannot install langpack packages in its next run as this happens only via hooks. So, I think now only and only thing remained is hook in dnf similar to postresolve hook.

Here is more details on Anaconda usecase:
1) Those people who prefer to see their desktop installed in their own local language need some way in Anaconda only so that when system starts after installation wherever language translation packages are available those applications will start in that local language only.

2) When Anaconda was using yum, to have above thing working code is written for langpacks plugin in https://github.com/rhinstaller/anaconda/blob/anaconda-21.48.22.19-1/pyanaconda/packaging/yumpayload.py#L250

This creates a file /etc/yum/pluginconf.d/langpacks.conf and writes the local language in which user want to see desktop installed.

3) The langpacks.conf file also stores a list of extra languages enabled in Anaconda other than default language of the desktop. This language list is used to generate a list of all language translation packages where a base package is already selected in anaconda installation transaction.

4) Anaconda then add this translation packages list via yum's postresolve hook to originally selected environment packages and perform complete installation of all such packages.

5) The other use of langpacks.conf is that one can simply add new language there and have its language translation packages picked for installation in next run of yum update command where base package is selected for updation. e.g. You added language fr in langpacks.conf and in the next if libreoffice update is available then libreoffice-langpack-fr package gets installed.

Note this functionality works but not included in yum-langpacks source code which I plan to add in dnf-langpacks as new enhancement.

6) So, all such addition of langpack packages happens in postresolve hook of yum which is missing in dnf.

If yum has not shown any issues in last few years by re-resolving transaction in postresolve hook, I wonder why should dnf fail/show broken dependencies?

Also, this functionality will remain to langpacks plugin only. Let's assume if dnf started showing broken dependencies by re-resolving transaction then only langpacks plugin user will get affected. I don't see a huge impact on users.

Comment 17 Pavel Odvody 2015-08-24 12:35:28 UTC
Rich dependencies will be disabled for core Fedora 23, but we'll try to roll an initial support for COPR repositories during the same timeframe.

Comment 18 Parag Nemade 2015-11-25 05:24:07 UTC
Jan,
   Can you Please help adding required hooks? I am getting few bugs where people are either using Fedora Server DVD or Fedora Workstation Live but as there is no hook available, evenif anaconda writes the selected language list to /etc/dnf/plugins/langpacks.conf file, langpacks plugin unable to read that list and add required langpack packages in first installation by anaconda. Thus, user complains why they can't get langpacks even after installing desktop in their own language.

Comment 19 Parag Nemade 2015-12-09 11:22:28 UTC
Ping DNF developers...

Comment 20 Honza Silhan 2016-01-04 15:47:27 UTC
I am sorry for the delay. So far I've come up with these solutions:

1) using weak deps + rich deps
This would require packaging changes + defining proper guidelines + (rpm macros)
- add additional metadata packages one per each lang to fedora repo, e.g. langpack-en. They will could replace langpack plugin config file
- for each langpack subpackage in each package: add "Supplements: %{name} AND langpack-<lang>". The core logic of langpack could be removed then when every package follow this guide line.

2) install the langpacks additionally with an extra command (refresh?)
i.e.: "dnf install libreoffice-core && dnf langpack refresh"
it will not passively install langpacks during dnf install command by checking installed packages and installing/removing to them langpacks

3) add DNF hook after resolution + additional resolution afterwards
Based on result of the first resolution you add additional langpack packages and the second depsolving would confirm the transaction. Note: no package would be put into transaction set directly. The dowside of this is that the result set could change noticeably in second resolution run and then irrelevant packages could be installed.

I still prefer doing just one depsolving. The best would be to do 1) when there are only a small amount of langpacks available. If you think that the maintainers would not honor the guide then we could find another solution.

Comment 21 Parag Nemade 2016-01-05 06:38:16 UTC
Here is what I think

1) This solution requires a lot of people involved, lots (141) of new packages (langpack-<lang>) to be added in Fedora. 

Will this work when we install using anaconda? When libreoffice-core package gets selected for installation and we already have selected few languages in anaconda for installation, will this first installation include those libreoffice langpack packages also?

when you want to include langpack-<lang> in transaction, will that include all the packages listed in Requires: for that langpack-<lang> metapackage? if yes then we really don't want that. 

If I am getting this wrong here, can you just suggest some example for me to try (modifications to existing libreoffice and hunspell package). I can spin a live iso and try to install in any non-english language and check if I can get libreoffice and hunspell language packages or not?

2) Here end user will see his regular usage is changed now when compared to yum-langpacks and confusing to him to run two commands to get a single package with its langpack packages.

3) Can you help me to understand why is it so difficult to re-resolve transaction? If I am not wrong then yum was doing the same right that is why it worked with yum-langpacks.


Whichever solution be chosen, I need following to work.
1) With anaconda installation, the number of languages getting installed, all should be documented somewhere and their respective language packages should be already installed when a user starts his system first time. There should be option available for end user to remove those language support later on.

2) There should be option to add extra language /remove installed language later on. There should be option to list installed languages.

3) Metapackages like say langpack-cs should not install all the available cs langpacks packages but only required packages.

Comment 22 Honza Silhan 2016-01-11 14:21:02 UTC
Here is the showcase of weak + rich deps solution for langpacks:

# dnf copr enable jsilhan/langpacks

// anaconda langpack selection
# dnf install tl-langpacks-cz
...
Installed:
  tl-langpacks-cz.x86_64 1-3         

Complete!

// anaconda installation of libreoffice, tl-libreoffice-core-langpacks-cz is pulled in
# dnf install tl-libreoffice-core
...
Installed:
  tl-libreoffice-core.x86_64 1-3       
  tl-libreoffice-core-langpacks-cz.x86_64 1-3     

Complete!

// adding another language + langpacks of this lang for already installed packages
dnf install tl-langpacks-es
...
Installed:
  tl-langpacks-es.x86_64 1-3           
  tl-libreoffice-core-langpacks-es.x86_64 1-3

Complete!

// installing another package, all langpacks are getting installed too
dnf install tl-hungspell
...
Installed:
  tl-hungspell.x86_64 1-3              
  tl-hungspell-cz.x86_64 1-3           
  tl-hungspell-es.x86_64 1-3           

Complete!

// removing language + all langpacks of this language
dnf remove tl-langpacks-cz
...
Removed:
  tl-hungspell-cz.x86_64 1-3           
  tl-langpacks-cz.x86_64 1-3           
  tl-libreoffice-core-langpacks-cz.x86_64 1-3

Complete!

// removing package with all related langpacks
dnf remove tl-libreoffice-core
Removed:
  tl-libreoffice-core.x86_64 1-3       
  tl-libreoffice-core-langpacks-es.x86_64 1-3

Complete!

Parag, let me know whether this is sufficient for you.

Comment 23 Parag Nemade 2016-01-12 07:01:54 UTC
I want to know how will anaconda know that if user selected cs, en_GB, pt_BR , zh_CN languages then select corresponding langpacks-cs, langpacks-en_GB, langpacks-pt_BR, langpacks-zh_CN packages in the transaction set?

Comment 24 Honza Silhan 2016-01-12 14:14:21 UTC
Anaconda should know the convention that the package is called "langpacks-<lang>", where <lang> is language string selected. With each selected language it will call "dnf install langpacks-<lang>" or from Python API "base.install('langpacks-<lang>')". It could be called as different transaction before / after the kickstart installation or at the same one. langpacks-<lang> metapackages should be put into fedora repo for each language in order to work.

Comment 25 Honza Silhan 2016-02-22 22:08:51 UTC
The last report (comment 6) was about Anaconda supporting langpacks in DNF backend. The new guideline for langpacks using rich and weak dependencies of RPM will be effective soon [1] and Anaconda has the support already [2]. Closing this. Please, open another RFE for plugin hook in new bug report, when needed.

[1] https://fedorahosted.org/fpc/ticket/593#comment:18
[2] https://github.com/rhinstaller/anaconda/pull/484


Note You need to log in before you can comment on or make changes to this bug.